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RouterStation 


Featuring a fast 6|OMHz MIPS 24K CPU, 64MB RAM, and 
16MB Flash; RouterStation provides a excellent horsepower 
for a variety of processor intensive multi-radio system 
applications. 


Up to 3 mini-PCl radios, 3 10/100 ethernet interfaces, a 5A 
power supply for multiple hi-power card support, USB 2.0, 
and enhanced temperature operating performance and 
ethernet ESD protection for carrier applications. 


MSRP $59 


Prices in USD. Ubiquiti Networks, Inc. Copyright © 2009 All Rights Reserved. 
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RouterStation Pro 


In response to the outstanding demand for our initial Rou 
OEM platform, Ubiquiti Networks announces the | 
Pro. Breakthrough Price/Performance with a $79 L 


Pro Version Enhancements: 

e A8V 802.3af Power Over Ethernet 

e 4-Port Gigabit Ethernet Switch 

e 256MB RAM 

e On Board SDIO Support 

e On Board, USB 2.0, RS232/dB9, and DC power jacks 


MSRP $79 
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First 6 months 
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*Offers begin September 1, 2009. Setup fee, minimum contract term, and other terms and conditions may apply. Visit www.1and1.com for full promotional offer details. Program 
and pricing specifications, availability and prices subject to change without notice. 1&1 and the 1&1 logo are trademarks of 1&1 Internet AG, all other trademarks are the property 
of their respective owners. © 2009 1&1 Internet, Inc. Alll rights reserved. 
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IXsystems is proud to introduce the latest offering in our ixX-Gemini 
line, the Gemini’. Cleverly disguised as any other 2U server, the Gemini’ 
secretly houses 4 highly efficient, extremely powerful RAID 5 capable servers. 
Each node supports the latest Intel® 5500 series processors, up to 
48GB of DDR3 memory, and three 3.5” hot-swappable hard drives. 


This system architecture achieves breakthrough x86 server 
performance-per-watt (375 GFLOPS/kW) to further satisfy the 
ever-increasing demands for efficiency, density and low-TCO 
of today’s high performance computing (HPC) clusters and 
data centers. For more information and pricing, please visit 
our website at http://www.iXsystems.com/gemini2. 
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Four hot-pluggable systems (nodes) ina 
2U form factor 


Each node supports the following: 


+ Dual 64-Bit Socket 1366 Quad-Core or Dual-Core Intel® Xeon® 
Processor 5500 Series 


+ 3x 3.5”SAS/SATA Hot-swappable Drive Bays 

Intel® 5520 Chipset with QuickPath Interconnect (QPI) 

+ Up to 48GB DDR3 1333/1066/800 SDRAM ECC Registered 

Memory 

1 (x16) PCI-E (Low Profile) 

+ Matrox G200eW 8 MB DDR2 Memory Video 

Integrated Remote Management - IPMI 2.0 + IP-KVM with 

dedicated LAN 

+ All four nodes share a Redundant 1200W High-efficiency Power 
Supply (Gold Level 92%+ power efficiency) 


. 


. 


800-820-BSDI Powerful. 
http://www.iXsystems.com Intelligent 


Enterprise Servers for Open Source 


Intel, the Intel logo, and Xeon Inside are trademarks or registered 


trademarks of Intel Corporation in the U.S. and other countries. 
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Next Month: INFRASTRUCTURE 


Everybody loves infrastructure, even the government—if you consider that 
keeping good company. If not, at least you're keeping company with all the 
Linux users out there building up the Internet and corporate infrastructure, 
making everyone safe from those other operating systems. 


To assist you in the process, next month's Infrastructure issue brings you articles 
on Storage Area Networks (SAN) on Linux, Host Identity Protocol for Linux 
(HIPL) and the Advanced Message Queuing Protocol (AMQ/AMQP). 


But wait, that’s not all. If your Infrastructure plans go beyond the mere 
terrestrial, don’t miss our article on IBM's InfoSphere Streams and Uppsalla 
University’s space weather project. Now that's data processing with a capital D! 
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With Next Generation Microway Solutions! 


TriComX Teraflop GPU Computing 
~ QDR/DDR InfiniBand HCA For Workstations and HPC Clusters 

~ ConnectX™ Technology ~ NVIDIA® Tesla™ GPU with 240 Cores on One Chip 

~ 1usec Latency ~CUDA™ SDK 


~ Switchless Serial Console ~ NVIDIA® Quadro® Professional Graphics 
~ NodeWatch™ Remote Management ~ AMD® FireStream™ GPU 
- Stream SDK with Brook+ 


Headers to fans, voltages, 
SOS MEM Gintedaceiand temperatures, On/Off and reset 
serial console switch 


RS-485/422 Daisy 
chain connectors 


InfiniBand or 
10GigE connector Mellanox® ConnectX 
InfiniBand HCA 
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Large Memory Scalable SMP Server 
~ Scales to 1 TB of Virtual 
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Shared Memory FasTree xX 
_ Up to 128 CPU Cores ~ Mellanox? InfiniScale™ IV Technology 
~ 8U System Includes ~ QDR/DDR InfiniBand Switches 


coos i 32 Quad Core CPUs 


~ Modular Design 
~ QDR 1 usec Backplane 


_ 4 GB/sec Bandwidth per Port 
~ QSFP Interconnects 


~ InfiniScope™ Real Time Diagnostics 


Call the HPC Experts at Microway to Design Your Next 
High-Reliability Linux Cluster or InfiniBand Fabric. 
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HOW MUCH VMWARE 


NAS-to-NAS mirroring. 


DO YOU NEED? 
VMware Certified NAS and Servers 


The scalable AberNAS is the only Network Attached Storage appliance in its class to carry the VMware® Ready 
Certification. Aberdeen enables VMware-centric organizations of any size to affordably take advantage of 
AberNAS’s robust features such as; superior RAID controller speed, increased scalability and optional 


VMware Certified Aberdeen Stirling servers offer a lower cost of ownership, while eliminating concerns of 
compatibility complications. Stirling servers possess exceptional benefits designed to deliver improved 
operational efficiency within a dynamic VMware environment. 
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ABERNAS 


VMware Certified NAS Up To 100TB 
* Quad-Core Intel® Xeon” Processor 5500 Series 

* 3GB/6GB 1333MHz DDR3 memory 

Supports both SAS & SATA Storage Drives 


¢ RAID 0, 1, 5, 6, 10, 50, 60 Capable 

« SAS Expansion Ports & iSCSI Target Capable 

* Dual/Quad Gigabit Ethernet Ports 

¢ 5-Year Warranty 
1U - 8TB NAS Starting at............. 54,995 
2U - 24TB NAS Starting at.......° 10,495 
3U - 32TB NAS Starting at.......° 12,995 
5U - 48TB NAS Starting at.......° 17,995 
6U - 64TB NAS Starting at....... °23,495 
8U - 100TB NAS Starting at..... °33,995 
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1U Dual Xeon Server 

Up to 2x Quad-Core Intel” 
Xeon® Processor 5400 Series 

* 32GB FBDIMM max memory 

Up to 4 SATA or SAS drives 

* 650W redundant power 

« Pre-installed VMware® ESXi 
on Disk-on-Module 

¢ 5-Year Warranty 


Starting at ad | ,845 


1U Twin Node Server 

Up to 2x Quad-Core Intel 
Xeon Processor 5400 Series 

* 64GB FBDIMM max memory 

Up to 2 SATA drives 

© 980W high efficiency power 

¢ Pre-installed VMware ESXi 
on Disk-on-Module 

¢ 5-Year Warranty 


Starting at $2,675 


2U Xeon Server 


* Single Quad-Core Intel Xeon 
Processor 5500 Series 

¢ 24GB DDR3 max memory 

Up to 6 SATA drives 

* 400W high efficiency power 

¢ Pre-installed VMware ESXi 
on Disk-on-Module 

 5-Year Warranty 


Starting at 54 7,495 


* Up to 4x Six-C 
Processor 730 
* 192GB FBDIM 


* 1200W redun 
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2U Quad Xeon Server 


* Up to 6 SATA or SAS drives 


¢ Pre-installed VMware ESXi 
on Disk-on-Module 
 5-Year Warranty 
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$5,275 
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2U Dual Xeon Server 
¢ Up to 2x Quad-Core Intel 
Xeon Processor 5500 Series 
¢ 144GB DDR3 max memory 
* Up to 8 SATA or SAS drives 
¢ 720W redundant power 
« Pre-installed VMware ESXi 
on Disk-on-Module 
 5-Year Warranty 


Starting at 52 ,095 
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4U Quad Xeon Server 
* Up to 4x Six-Core Intel Xeon 
Processor 7300 Series 
* 192GB FBDIMM max memory 
* Up to 5 SATA or SAS drives 
¢ 1200W redundant power 
* Pre-installed VMware ESXi 
on Disk-on-Module 
 5-Year Warranty 


Starting at 55 7345 


888-297-7409 


www.aberdeeninc.com/]j031 
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SHAWN POWERS 


The Kyle Rankin Issue 
(He Wishes!) 


ack when we were creating the editorial 
34 calendar for 2009 and decided on “Hack 

This” as the topic for October, | think our 
resident Hack & / columnist Kyle Rankin thought 
that issue was going to be dedicated completely 
to him. (Either that, or he was a bit worried we'd 
make him write every single article in the issue.) 
As it turns out, neither was true. | considered 
telling him we needed enough content from him 
this month to fill the magazine, but to be hon- 
est, Kyle scares me a bit. So, | saved my joking 
for this piece. 

Just because our Hack Editor didn’t write 
every jot and title this month doesn’t mean we 
won't leave as better hackers. Duilio J. Protti gets 
us going with low-level system programming, 
but we don’t use an actual system. Thanks to 
KVM, we learn low-level system hacking on a vir- 
tual machine. Virtual machines are so robust 
nowadays, the concepts and practices will work 
on a real system too. If software hacking isn’t 
enough for you, fear not. Marco Fioretti shows 
us some actual hardware hacking with Field 
Programmable Gate Arrays. You won't need to 
break out a soldering iron, but it'll just about feel 
like it! 

If all this system hacking sounds fun, but 
you're worried your proprietary computer BIOS 
will hinder your skills, Anton Borisov might be 
just the man you want to listen to. He shows us 
the ins and outs of coreboot (you may know it 
as LinuxBlOS). Although flashing the firmware on 
off-the-shelf routers might be second nature, not 
too many of us have ventured into BIOS flashing. 
It seems a perfect fit for this issue. 

If you've hacked your computer so that it 
bends to your every whim, you definitely want it 
to look cool as well. Sure, Compiz on your desk- 
top will draw oohs and ahhs from the occasional 
Windows user, but even Linux users will be 
impressed if you implement Clutter in your pro- 
gramming. Clutter allows you to add rich, GUI 
interfaces to your applications quickly and easily. 
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Alex Crits-Christoph demonstrates utilizing 
Clutter’s advanced toolkit. In fact, we've got a 
bunch of programming hacks this month. 
Reuven M. Lerner talks more about testing with 
RSpec for Rails, and Dave Taylor shows us some 
cool (and $RANDOM) Web server tricks. 

And, of course, the “Kyle Rankin” issue 
wouldn’t be complete without his monthly Hack 
&/ column. This time, Kyle shows us his secret 
to fighting spam (and its only slightly more palat- 
able cousin, e-mail ham) preemptively. We all 
have a handful of “throw-away” e-mail address- 
es we use when signing up for something on- 
line, but Kyle uses a different address every sin- 
gle time. So while Kyle shows us how to keep 
unwanted things out of our inboxes, his buddy 
Bill Childers demonstrates how to put things we 
might want (namely Android) in some bizarre 
places. A few months back when the editorial 
team was talking about content for this issue, | 
challenged him to install Android on his 
Netbook. He rose to the challenge, and not only 
got Android running on his Netbook, but also on 
a Windows Mobile phone! 

We have a full lineup of articles on other top- 
ics as well. Whether you are looking for desktop 
security tips from Mick Bauer, new product infor- 
mation from James Gray or a rally cry to take 
over the Internet from Doc Searls, you're gonna 
love his issue. Heck, even | get into the hacking 
act with Google Voice. You might be able to get 
unlimited minutes on your cell phone thanks to a 
handy little trick with your free GV phone num- 
ber. So for this month, we all got to be hacks. 
You can too. And if you become a hacker, don’t 
worry, we won't tell Kyle. We wouldn't want him 
to get jealous.m 


Shawn Powers is the Associate Editor for Linux Journal. He's also the Gadget 
Guy for LinuxJournal.com, and he has an interesting collection of vintage 
Garfield coffee mugs. Don’t let his silly hairdo fool you, he’s a pretty ordi- 
nary guy and can be reached via e-mail at shawn @linuxjournal.com. Or, 
swing by the #linuxjournal IRC channel on Freenode.net. 
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1U INTEL® XEON® PROCESSORS 5500 SERIES 
SERVER W/ 4X 3.5" HOT-SWAP SATA DRIVE BAYS 


* Supermicro 1U Rackmount Server with 560W Power Supply 

* Supermicro Server Board w/Intel® 5520 Chipset 

* Support up to Dual Intel® 5500 series Xeon® 
Quad/Dual-Core, with QPI up to 6.4 GT/s 

* Support up to 96GB DDR 1333/ 1066/ 800MHz ECC 
Reg.DIMM 

* 4x 3.5" Hot-swap SATA Drive Bays 

* Intel® 82576 Dual-Port Gigabit Ethernet Controller 
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SDR-S4313-T24 
STARTING a Ot, 899 


4U INTEL® XEON® PROCESSORS 5500 SERIES 
SERVER W/ 24X 3.5" HOT-SWAP SAS/SATA BAYS 


* Supermicro 4U Rackmount 9O00W (1+1) Red. Power Supply 

* Supermicro Server Board w/ Dual Intel® 5520 Chipsets 

* Support up to Dual Intel® 5500 series Xeon® Quad/Dual- 
Core, with QP! up to 6.4 GT/s 

* Support up to 144GB DDR3 1333/ 1066/ 800MHz ECC 
Reg. DIMM 

* 24x 3.5" Hot-swap SATA Drive Bay 

* Intel® 82576 Dual-port Gigabit Ethernet Controller 


1.877.727.7886 / www.ServersDirect.com 


"5 $809 


SDR-S1341-T00 is among our most cost-effective 1U Xeon 
Servers, and it is ideal for large high-performance computing 


| SDR-S1343-T04 


The ServersDirect® Systems with the Intel\® Xeon® Processor helps you simplify computing operations, accelerate performance and 


virtualization. 
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2U INTEL® XEON® PROCESSORS 5500 SERIES 
SERVER W/ 8X 3.5" HOT-SWAP SAS/SATA BAYS 


* Supermicro 2U Rackmount Server with 560W Power Supply 

* Supermicro Server Board w/Intel® 5500 Chipset 

¢ Support up to Dual Intel® 5500 series Xeon® 
Quad/Dual-Core, with QPI up to 6.4 GT/s 

* Support up to 24GB DDR3 1333/ 1066/ 800MHz ECC 
Reg.DIMM 

* 8x 3.5" Hot-swap SATA Drive Bays 

* Dual Intel® 82574L Gigabit Ethernet Controller 
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3U INTEL® XEON® PROCESSORS 5500 SERIES 
SERVER W/ 16X 3.5" HOT-SWAP SAS/SATA BAYS 


* 3U Rackmount Server with 1+1 900W Red. Power Supply 

* Supermicro Server Board w/ Dual Intel® 5520 Chipsets 

¢ Support up to Dual Intel® 5500 series Xeon® Quad/Dual- 
Core, with QPI up to 6.4 GT/s 

¢ Support up to 96GB DDR3 1333/ 1066/ 800MHz ECC 
Reg.DIMM 

* 16x Hot-swap SAS/SATA Drive Bays 

Intel® Dual 82576 Dual-Port Gigabit Ethernet (4 ports) 


SERVERS DIRECT CAN HELP YOU CONFIGURE YOUR NEXT HIGH PERFORMANCE SERVER SYSTEM - CALL US TODAY! 


Our flexible on-line products configurator allows you to source a custom solution, or call and our product 
experts are standing = to help vou to ree systems that require a little extra. Servers Direct - your direct 


Intel, Intel logo, Intel Inside, Inte! Inisde logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, Pentium, 
and Pentium II| Xeon are trademarks of Intel Corporation or it’s subsidiaries in the United States and other countries. 


APPLICATION SERVER 


Refresh your servers with new SDR-S1337-T02 powered by 
Intel® Xeon® processor 5500 series, based on intelligent 
performance, automated energy efficiency and flexible 


SDR-S2311-T08 


GO STRAIGHT TO THE SOURCE! 
1.877.727.7887 | www.ServersDirect.com 
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PEDESTAL INTEL® XEON® 
PROCESSORS 5500 SERIES 
SERVER W/ 10X HOT-SWAP 
(OPT.) SATA BAYS 


* Intel Pedestal Chassis w/ 750W (1+1) Power Supply 

* Supermicro Server Board w/Intel® 5520 Chipsets 

* Support up to Dual Intel® 5500 series Xeon® 
Quad/Dual-Core, with QP! up to 6.4 GT/s 

* Support up to 96GB DDR3 1333/ 1066/ 800MHz ECC 
Reg,/unbuffered DIMM 

* Option 10x 3.5" Hot-swap SATA Bays 

* Intel® 8257EB Dual-port Gigabit Ethernet Controller 
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9U INTEL® XEON® PROCESSORS 5500 NEHALEM 
SERIES SERVER W/ 50X HOT-SWAP SATA II / SAS 
BAYS 

* QU Chassis with 1620W Redundant Power Supply 


* Supermicro Server Board w/ Dual Intel® 5520 Chipsets 

* Support up to Dual Intel® 5500 series Xeon® 
Quad/Dual-Core, with QP| up to 6.4 GT/s 

* Support up to 144GB DDR3 1333/ 1066/ 800MHz 
ECC Reg. DIMM 

* 50 x 3.5"Internal SATA Drives Trays 

* Intel® 82576 Dual-port Gigabit Ethernet Controlle 
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Twitter 

I'm with Kyle Rankin on Twitter, 

and what is more, | have tried it 

[see Kyle Rankin and Bill Childers’ 
Point/Counterpoint debate on Twitter in 
the August 2009 issue]. As far as | have 
found, at least 90%, probably 99%, of 
content is pointless and uninteresting, 
even from the few people that | know. 
Reading the drivel of strangers is even 
more time-wasting. What | am doing 
now is usually private and personal— 
not even for sharing with friends, let 


alone with, potentially, the whole world. 


Several years ago, | gave up all forms of 
instant messaging. | am not prepared to 
react instantly to the needs of others. If 
people want to contact me, they can 
send me e-mail, and | will reply when | 
feel like it. There is no need for any- 
thing to be instant. | stopped answering 
the telephone six years ago and have 
not died from the lack of instant 
contact. In fact, it is quiet peaceful. 


Rhian Geleick 


| envy your ability to shut out the “on 
call” structure so many of us are 
required to live out. That said, Twitter 
either can be useful or it can be a time 
suck. The same thing can be said for 
tons of everyday things we use: cell 


phones, e-mail, instant messaging, 
books, food and so on. I’m not suggest- 
ing a person would die without Twitter 
(unlike with food), but | say, if it’s 
useful, use it. If not, don’t.—Ed. 


Dropbox 

| just received my August 2009 LJ and 
noticed that next month, Cross-Platform 
Development is the topic. | recently 
came across a cool app that helps 
share files across cross-platform OSes 
without a hitch. It is called Dropbox 
(www.getdropbox.com). They have 
installs for Linux, MS and Apple. | 
actually can share files in my Virtual 
Machine with my host, without having 
to make sure that the file shares are set 
up properly. 


They offer a 2GB account for free sub- 
scribers, and you can access your files 
from their “cloud” from any computer 
that has Internet access. For larger 
groups, they have options for paid 
accounts with larger space provisions. 


You guys are still the highlight to my 
trips to the mailbox each month! Keep 
it up! 


soosurfer 


You're absolutely right. Dropbox is one 
of those nifty apps that is indeed cross- 
platform and largely relies on Internet- 
hosted services. It's not quite a Web-based 
app, but leverages the Internet as a 
means to communicate. We demonstrat- 
ed the virtues of Dropbox on our Web 
site as well: www.linuxjournal.com/ 
video/dropbox-linux. /t’s an awesome 
tool!—Ed. 


Using banner Instead of motd 
for Legal Text 

In the article “Right Command, 
Wrong Server” [July 2009], Kyle Rankin 
mentioned adding a legal warning to 
the message of the day to be delivered 
to users after they log in. For legal 
warnings, | believe the sshd banner 
option would be more appropriate. The 
banner is presented to users before a 
successful login, so people are warned 
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well before they gain access to the 
system. Edit the /etc/ssh/sshd_config (or 
distro-specific sshd_config location), and 
uncomment the following line: 


Banner /etc/issue.net 


Then create the text file (/etc/issue.net) 
that contains your legal warnings to 
be presented to users before they even 
attempt to log in. Remember to restart 
sshd to apply the change. 


Kevin Benton 


Kyle Rankin replies: Thanks for the 
tip! IANAL, so I’m not even sure how 
effective these notices are in a court 
of law. | recommend checking with a 
lawyer to see how enforceable these 
notices are before you do a system- 
wide deployment. 


Help with MyBook Hack 

lam an LJ subscriber, and | was so 
inspired by Federico Lucifredi’s article 
“Hacking Your Portable Linux Server” 
in the July 2009 issue that | searched 
eBay until | was able to buy my 

own WD MyBook WE NAS drive 
[MDL:WD10000G032-001]. On eBay, 

it was advertised as WDG1NC10000N 
World Edition 1TB. | don't know yet if 
there is any subtle difference between 
these models. It arrived three days 

ago, and without trouble, my browser 
brought up the WD Shared Storage 
Manager Web interface, reporting 
firmware version 2.00.15. As an 
upgrade was available, | immediately 
upgraded to 2.00.18. | easily followed 
your article's instructions and used 
Martin Hinner’s script to set up SSH and 
then log in from a terminal on my PC 
running Linux Mint. (Shared Storage 
Manager gave the “update has failed” 
message that you mentioned, as expected 
with this latest firmware.) | switched 

to root, made ssh permanent through 
inittab, and disabled MioNet (line 19 in my 
post_network_start.sh) and also verified 
that my kernel was 2.6.17.14. | logged 
out, rebooted, logged back in, and yes, 
ssh was permanent. So far so good. 
Martin Hinner also provided instructions 


to install nano, an alternative text editor 
to vi, so | used wget to obtain the ncurses- 
5.7 (required by nano) and nano-2.09 
tarballs and untarred them. When | tried 
./configure for ncurses-5.7, | hit a 
wall! It reported “gcc no, cc no, no 
acceptable cc found in $PATH”. 


As | understand it, the problem is (at 
least) that there is no gcc compiler on 
this MyBook. | have checked almost 
every directory, and it’s certainly not in 
/usr/bin or /usr/sbin or anywhere else that 
| can see. So my question is, how can | 
install a gcc ARM9 compiler? | will need 
to obtain it as a compatible binary and 
install it, so gcc source doesn’t help me 
unless | need to first cross-compile on my 
Linux Mint PC and then transfer from 
there. This is less important, but it would 
be interesting to know: when did WD 
stop including gcc? If | hadn’t upgraded 
from 2.00.15 to 2.00.18, would the 
compiler still have been there? Could 
it be a subtle difference between 

my WD10000G032-001 and the 
WDG1NC10000N? Could Martin Hinner’s 
scripts or anything else | did have deleted 
gcc? | am stuck and can’t make any 
progress until | get a working compiler. 


Your article mentioned the presence of 
gcc3.4.2, gmake, wget and so on, and 
it gave output for df -h (same as mine) 
and cat /proc/mdstat (the same 
except for md4, as mine is a single disk 
1TB). Everything seems to check out 
except the presence of the gcc compiler. 
Any help with my problem would be so 
appreciated, and a follow-up MyBook 
article also would be great. 


JR 


Federico Lucifredi replies: The 
WDG1NC10000N is a single-drive 
“Blue LED” WD Mybook World 
Edition, and it has hardware matching 
what is described in the article. 


Your trouble arises from the 2.x 
firmware. Although the 1.x firmware 
versions include the full GCC toolchain, 
| recently upgraded one of my devices 
and found that GCC is missing in the 
more recent firmware. 


| ignore the reason for the change (indeed, 
Western Digital endorsed direct access 
to Linux in the latest “White LED” devices, 


(LETTERS) 


where root ssh access can be configured 
straight off the Web interface), but you 
have two options for recovery: heading 
over to the MyBook Community Wiki 
(mybookworld.wikidot.com/forum/ 
t-50241/tutorial:how-to-recover-gcc- 
after-firmware-2-00-15-upgrade) for 
recovery instructions would be the first. 
The alternative is for you to build a 
cross-compiling toolchain, which is more 
involved but certainly convenient, as this 
way you can build your software for 
beefier machines. If you choose this latter 
approach, O’Reilly’s Building Embedded 
Linux Systems is your best introduction. 


| also want to note that ipkg 
(mybookworld.wikidot.com/optware) 
is on its way to the MyBook. Look for 
prebuilt packages before you rebuild 
everything yourself. 


There is certainly more that can be said 
on the subject. | will consider a follow-up 
article if the interest is there. 


Squid Correction 

I've been waiting for the last installment 
of Mick Bauer's “Building a Secure 
Squid Web Proxy” series [see the May 
through August 2009 Paranoid Penguin 
columns] since it was announced back 
in the April issue. Linux Journal is one 
of the magazines | read from front to 
back, and | enjoy it. 


| noticed a couple errors in the last article 
of the series [August 2009]. One is in the 
instructions to install the blacklists with 
the --strip argument for the tar com- 
mand. The code states --strip 1 and the 
explanation states --strip 2. The other 
error is in the “Configuring Squid to Use 
squidGuard” section at the end of the 
fourth paragraph. The last word should 
be changed from url_rewrite_program 
to url_rewrite_children. 


One more feature that could be added 
to this series is how to configure squid 
as a transparent proxy. | think that a lot 
of us use Linux or other *nix systems as 
a firewall to protect home or other net- 
works. When using a transparent proxy, 
there is no need to configure browsers. 
There is also protection from other 
applications that might be accessing 
banned sites. Keep up the good work. 


jschiavon 
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Dracula Correction 

Your pipeline of tr and grep has the unintended consequence of eliminating 
any words that contain, are followed, or are preceded by punctuation [see Dave 
Taylor's “Looking More Closely at Letter and Word Usage” in the August 2009 
issue]. This is not an insignificant number, as it eliminates the last word of 
every sentence and comma-separated clauses, plus the first and last word 
of quotations, contractions and hyphenated words. 


Using the alternative pipeline, | mentioned a few issues back [see the July 2009 
Letters section]: 


' 


[:lower:]' | tr -cs '[:lower:]' '\n' 
the word count of the Dracula text increased by more than 20%. 


However, this pipeline results in a different error. Contractions, such as o'clock 
and didn’t, get split into multiple words. The Dracula text contains many unusual 
contractions such as y’are, Yell, a’hidin’ and gard’ners. Some strange words 
result when they are split. 


Jon 


Dave Taylor replies: You're talking about this code snippet: 


$ cat. dracuila.txt. | tr * * "\ 
' | grep -v '[4[:alpha:]]' | grep -v "%$" 


And you're exactly right. | can’t believe | didn’t notice that! Thanks for the 
correction, Jon! 


PHOTO OF THE MONTH 


Have a photo you'd like to share with LJ readers? Send your submission to 
publisher@linuxjournal.com. If we run yours in the magazine, we'll send you a free T-shirt. 


Bringing Tux Along for the Ride—Submitted by Bill Parducci 
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PRINT SUBSCRIPTIONS: Renewing your 
subscription, changing your address, paying your 
invoice, viewing your account details or other 
subscription inquiries can instantly be done on-line, 
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PDFs anywhere in the world for one low cost. 
Visit www.linuxjournal.com/digital for more 
information or use the contact information above 
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LETTERS TO THE EDITOR: We welcome 
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them at www.linuxjournal.com/contact or 
mail them to Linux Journal, PO Box 980985, 
Houston, TX 77098 USA. Letters may be edited 
for space and clarity. 


WRITING FOR US: We always are looking 
for contributed articles, tutorials and real- 
world stories for the magazine. An author's 
guide, a list of topics and due dates can be 
found on-line, www.linuxjournal.com/author. 


ADVERTISING: Linux Journal is a great 
resource for readers and advertisers alike. 
Request a media kit, view our current 

editorial calendar and advertising due 

dates, or learn more about other advertising 
and marketing opportunities by visiting us 
on-line, www.linuxjournal.com/advertising. 
Contact us directly for further information, 
ads@linuxjournal.com or +1 713-344-1956 ext. 2. 


WEB SITE: Read exclusive on-line-only content on 
Linux Journal's Web site, www.linuxjournal.com. 
Also, select articles from the print magazine 

are available on-line. Magazine subscribers, 
digital or print, receive full access to issue 
archives; please contact Customer Service for 
further information, subs@linuxjournal.com. 


FREE e-NEWSLETTERS: Each week, Linux 
Journal editors will tell you what's hot in the world 
of Linux. Receive late-breaking news, technical tips 
and tricks, and links to in-depth stories featured 
on www.linuxjournal.com. Subscribe for free 
today, www.linuxjournal.com/enewsletters. 


diff -u 


WHAT’S NEW IN KERNEL DEVELOPMENT 


The state government of New 
York is using git! The github.com/ 
nysenatecio site hosts GPLed tools 
launched by the New York State Senate, 
including open.nysenate.gov/ 
openleg, which lets users search 
through ongoing legislative activities. 
The project seems very kernel-centric, 
even using a MAINTAINERS file 
similar to that of the Linux kernel. 
The data at the back end of these 
tools also is freely available, so 
people can write their own front- 
end sites or contribute to the state's 
effort. Neat! 

The Xen developers are tasting 
the bitter pill of rejection. Many 
Xen users see a real need for Xen to 
be accepted into the kernel in the 
immediate future, but the kernel 
folks are holding out due to various 
problems. First, the Xen code is very 
invasive and apparently is not very 
respectful of the areas of the kernel 
it touches. As Linus Torvalds puts 
it, “Xen craps all over other people's 
code”, and elsewhere, he said, 
“Xen pollutes the architecture code 
in ways that no other subsystem 
does.” There also are certain features 
of Xen that people such as Alan 
Cox feel are badly implemented, 
to the point that they would create 
long-term maintenance and debugging 
problems for the kernel, which 
could not simply be backed out 
when better solutions arose. 

The issues standing in the way 
of Xen inclusion in the Linux kernel, 
therefore, are all messy, difficult 
issues that would require the Xen 
Project in some ways to come to 
a screeching halt and move off in a 
different direction. The Xen developers 
currently are of a single voice in 
resisting this, but there’s really no 
alternative. Their best bet probably is 
to submit small, sensible patches that 
make the kernel more and more nat- 
urally able to support Xen features. 


Alan Cox, Theodore Y. Ts’o, Ingo 
Molnar and others have expressed an 
interest in helping the Xen developers 
identify how best to direct their 
energies toward getting code into 
the kernel. But, just hoping that 
one day Linus decides Xen is too 
important to keep out of the kernel 
is traditionally a losing strategy. 

Once a crucial driver and the 
topic of endless horrifying flame 
wars, the IDE code is changing 
hands yet again, from Bartlomiej 
Zolnierkiewicz to David S. Miller, 
who says he will consider IDE to be 
“legacy code”. IDE is, in fact, grad- 
ually being obsoleted by the PATA 
drivers, but there still are plenty of 
users tied to the IDE code, and so 
bug fixes and other enhancements 
will continue to be important for 
some time to come. The linux-next 
tree already has switched from 
using Bart's tree to David's, main- 
tained at master.kernel.org:/pub/ 
scm/linux/kernel/git/davem/ 
ide-2.6.git. 

Guo Hongruan announced 
that he’d successfully gotten Linux 
compiled and running on the 
TriMedia microprocessor, used 
typically for audio and video pro- 
cessing. TriMedia has no GCC 
target, making the port a difficult 
one. Guo had to run the kernel 
through a preprocessor to translate 
all the special GCC extensions into 
standard C99 source code, after 
which it compiled under the tmcc 
compiler. His work is available at 
tmlinux.googlecode.com. 

Jon Masters has started a pod- 
cast of the linux-kernel mailing 
list, providing semi-daily summaries 
of events. He was inspired by the 
desire to force himself to keep up 
with the mailing list. Audio is avail- 
able at kernelpodcast.org, and 
there’s also an RSS feed available. 

—ZACK BROWN 
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| want all my code to be 
open source, but | will use 
the best tool for the job, and 
BitKkeeper was the best tool, 
and at the time the alterna- 
tives sucked so bad. When 
the alternatives are so bad, 

| will take proprietary code. 
Proprietary was a downside, 
but what choice did | have? 
Hey, | usually do my presen- 
tation slides in PowerPoint. 


—Linus Torvalds 


Looking at the proliferation 
of personal Web pages on 
the Net, it looks like very 
soon everyone on Earth will 
have 15 megabytes of fame. 


—M.G. Sriram 


Information on the Internet 
is subject to the same rules 
and regulations as conversa- 
tion at a bar. 


—George Lundberg 


Any sufficiently advanced 
bug is indistinguishable from 
a feature. 


—Rich Kulawiec 


A people that values its 
privileges above its principles 
soon loses both. 


—Dwight D. Eisenhower 


It’s always the good men 
who do the most harm in 
the world. 


—Henry Adams 


NON-LINUX FOSS 


ClamWin is an open-source antivirus program for Windows. ClamWin provides 

scheduled virus scans and automatic downloads of its regularly updated virus 

database. It also integrates with Explorer to provide right-click menu integration for 
scanning of 


@ ClamWin Preferences ; x| individual files 


or directories. 


Email Alerts | Archives | File Locations | Reports | Advanced | : 1 
General | Filters | Internet Updates | Proxy | Scheduled Scans | engl were 
an Outlook 
+ Scanning Options user, it pro- 
\ws es vides an add- 
I Display Infected Files Only in for remov- 
ing infected 
+ Infected Files attachments. 
@ Report Only ClamWin 
© Remove (Use Carefully) is based on the 
© Move To Quarantine Folder: open-source 
C:\Documents and Settings\All Users\. clamwin\quarantine ClamAV 
engine, an 
antivirus toolkit 
for UNIX and 
UNIX-like 
OK | Cancel | systems (as 
well as 
Windows sys- 


ClamWin Preference Pages (from www.clamwin.com) 


tems). ClamAV 
has built-in support for numerous types of archive files: tar, gzip, bzip2, zip, RAR, Cabinet, 
CHM and others. It also has built-in support for ELF and PE executables as well as 
compressed executables using numerous compression schemes. Many popular document 
formats also are supported: MS Office, Mac Office, HTML, RTF and PDF, among others. 
—MITCH FRAZIER 


LinuxJournal.com 


A little hack here and a little tweak there, and you have a new LinuxJournal.com. 

While working on the new version of the site you all know and love, | have 
thought about the word “hack” a lot. | think we in the Open Source community 
in particular have found “hack” to be a useful term with a broad definition. 
When working with software and platforms that are community supported, 
there is not always a “right” way to do things or a well-documented process. 
Thus, we find ourselves making our own recipes and hacks to get the job 
done whichever way we can. To me, a new way of laying out a Web page may 
require a workaround that | think of as a theming hack, or | may need to write 
or modify a Drupal module in order to add functionality to avoid hacking, as 
| always follow the first rule of Drupal, “don’t hack core!” Are these code 
modifications worthy of the word “hack”? | can't break in to a secure network, 
but | can (usually) find a solution to a Web development conundrum. 

Maybe we're all hackers then. As you read through this month's articles 
focused on hacks, | hope you discover a new way of solving your problems, 
a workaround that saves you some time, or even just a bit of inspiration 
that leads you to something new and different. Then, visit LinuxJournal.com 
to share your hacks with the rest of us! 

—KATHERINE DRUCKMAN 
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October 2009 


. Number of Google groups with more than 100,000 
members: 33 


. Number of Google groups that receive more than 
10,000 messages per month: 28 


. Number of Google groups that receive 0 messages 
per month: 4,331,894 


. Number of Google groups related to computers: 
92,681 


. Number of Google groups related to computers 
that receive 0 messages per month: 74,307 


. Percent of global Internet users that visit 
google.com (Alexa site rank #1): 32 


. Percent of global Internet users that visit 
yahoo.com (Alexa site rank #2): 26 


. Percent of global Internet users that visit 
youtube.com (Alexa site rank #3): 19 


. Percent of global Internet users that visit 
facehook.com (Alexa site rank #4): 19 


. Percent of global Internet users that visit live.com 
(Alexa site rank #5): 16 


. Number of articles in Linux Journals article 
database: 9,116 


12. Number of times the word Linux appears in LJ’s 
article database: 37,671 


13. Number of times the word Windows appears in 
LJ’s article database: 4,956 


14. Number of times the word kernel appears in LJ’s 
article database: 7,796 


15. Number of times the word KDE appears in LJ’s 
article database: 1,367 


16. Number of times the word GNOME appears in LJ's 
article database: 1,169 


17. Number of hits per month on linuxjournal.com 
from the Yahoo Web crawler: 335,310 


18. Number of hits per month on linuxjournal.com 
from the Google Web crawler: 310,270 


19. US National Debt as of 07/05/09, 15:30:12pm MST: 
$11,501,541,973,785.60 


20. Change in the debt since last month’s column: 
$97,726,931,237.70 


Sources: 7-5: Google | 6-10: www.alexa.com 
11: MySQL / 12-16: fmt -1 | sort | uniq 
17, 18: Drupal | 19: www.brillig.com/debt_clock | 20: Math 


www.linuxjournal.com october 2009 | 15 


(UPFRONT | 


Is Telnet Really the Root of All Evil? 


FOR QUITE SOME TIME, security experts have been claiming 
that Telnet is one of the worst protocols out there, and that it 
should be eradicated from the earth. According to these so- 
called experts, no one should use anything but SSH. Well, I’m 
here to show you that they are completely wrong. Telnet is 
one of the greatest gifts to humans. 

The above paragraph should ensure some rabid e-mail 
messages. For those of you still reading, | actually am serious 
about Telnet. (Note: | am not talking about telnetd.) The Telnet 
protocol is made up of two parts: the server portion, telnetd, 
and the client portion, Telnet. Now, | won't pretend to disagree 
with the general attitudes toward the server portion, telnetd, 
and its inherent insecurities. But, | also think we shouldn't 
throw out the baby with the bathwater. 

The client portion, Telnet, is a very useful tool in its own 
right. Say you're trying to configure your mail client and are 
running into problems. What options do you have? First, 
check to see whether the mail POP server is up. How? Why 
ping it, of course. Oh, but the IT staff drank the ICMP Kool-Aid 
and decided to block all ICMP traffic. So, now ping won't 
work. Are you stuck? Of course not—Telnet to the rescue. 

How can you use Telnet in this situation? Telnet isn’t limited 
to operating on the standard port. When you run Telnet, you 
can specify both a server and a port number. So, you can use 
it to try to connect to your mail POP server and check whether 
it’s up by using: 


telnet popserver.com 110 


In this example, you're trying to connect to the POP server 
in order to read your mail. As Telnet operates over TCP, block- 
ing ICMP packets shouldn't be troubling. Running the above, 
you should see a response like this (assuming that the mail 
server is up and running): 


Trying X%.X.KeXess 

Connected to popserver.com. 
Escape character is '“]'. 

+0K InterMail POP3 server ready. 


Now you can see that the mail server is up, so there must 
be another issue. This is where Telnet becomes really useful. POP 
is a text-based protocol, like many other Internet protocols. 
Because it is just text going back and forth, you can use the 
Telnet client to talk to the mail server. 

For the POP protocol, you simply can log in with your user 
name and password by using the following commands: 


USER username 
PASS password 
-ERR invalid user name or password 


With the above, you can check that you got the credentials 
right. As you can see in the example, the password is wrong. 
Trying again with the correct password gets you into the mail POP 
server. You can check whether you have mail with this command: 


16 | october 2009 www.linuxjournal.com 


LIST 
+0K @ messages 


You even can use Telnet to send e-mail. If you have access 
to a mail server, you can connect to port 25 and send e-mail 
using the following: 


telnet mailserver.com 25 

Trying X.X.X.X... 

Connected to mailserver.com 
Escape character is '4]'. 

220 mailserver.com ESMTP Postfix 
MAIL FROM: <user1@host1.com> 

250 2.1.0 Ok 

RCPT TO:<user2@host2.com> 

250 2.1.5 Ok 

DATA 

354 End data with <CR><LF>.<CR><LF> 
From: useri@host1l.com 

To: user2@host2.com 

Subject: Test 


Hello. This is a test of sending mail by telnet. 
Joey 


250 2.0.0 Ok: queued as 1DE8B2830E38 
QUIT 

221 2.0.0 Bye 

Connection closed by foreign host. 


POP and SMTP are not the only text-based protocols on 
the Internet. You can use Telnet to check Web sites too. Most 
Web servers operate on port 80. If you want to check connec- 
tion issues with a Web site, you can use this: 


telnet google.com 80 
And, you'll get a result like this: 


Trying 74.125.45.100... 
Connected to google.com. 
Escape character is '‘]'. 


The HTTP protocol also is text-based, so if you want a 
particular page, use the following command: 


GET index.html 

HTTP/@.9 200 OK 

Cache-Control: private, max-age=0 

Date: Mon, 06 Jul 2009 18:29:40 GMT 
Expires: -1 

Content-Type: text/html; charset=IS0-8859-1 
Server: gws 


<!doctype html><html><head>......... 


And, you will get the page back from 
the Web server. Telnet is the smallest and 
simplest mail client and Web browser, all 
wrapped into one. Of course, you need 
to have several Internet protocols memo- 
rized, but you really shouldn't be so lazy. 

Using this ability, you also can do 
some basic system probing. With nmap, 
another popular tool, you can get a list 
of the open ports on a host machine. 
Starting there, you can try connecting to 
each of the ports and seeing what kind 
of output you receive. | had to do this 
recently with a remote site where | was 
having difficulty connecting to the host 
through SSH. Using Telnet, | was able to 
see the initial response from the SSH 
server as the following: 


telnet host.com 22 
Trying A.X.K Xess 
Connected to host.com. 
Escape character is '4]'. 
SSH-1.99-OpenSSH_3.9p1 


This told me there was an issue with 
the particular version being run on the 
server, and | was able to go to the system 
administrator with an intelligent argument 
for upgrading the SSH server program. 

It’s still good to keep Telnet around— 
at least the client portion. Although the 
server portion has been considered a 
security risk in the past, the client portion 
can serve as a useful tool in network 
diagnostics. Sometimes old tools are 
the best, at least for some jobs. Have 
fun exploring your networks. 

—JOEY BERNARD 
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Calibre—Your eBook Librarian 


. aes As the proud owner of 
8 a = ey both a Sony PRS-505 
and a Kindle Dx, it's 
quickly come to my 
attention that most 
eBook software is 
designed for operating 
systems other than 
Linux. With the Kindle, 
Amazon Is trying 
to remedy that with 
computer-less converting 
over e-mail and such, 
but many of us want 
more local control. We 
also want to manage 
non-DRM books 
obtained from other 
vendors. Thankfully, 
the people behind the open-source Calibre fully support Linux with their 
Java-based program. Honestly, it's the best eBook management software 
available for any platform, so as Linux users, we’re not sacrificing anything. 
Calibre manages to do several things, and do them well: 


wating io in FI 


Calibre in Action (screenshot from calibre.kovidgoyal.net) 


@ It makes a sortable and searchable eBook library. 

@ It converts eBooks to formats supported by all major eBook readers. 
@ Books can be uploaded directly to the eBook readers. 

@ It automatically retrieves metadata and cover art. 


@ Blogs, newspapers and other on-line information can be downloaded 
automatically and converted to eBook formats. 


With all those features, Calibre still manages to be free and open. If you 
own an eBook reader or are considering buying one, check out Calibre today: 
calibre.kovidgoyal.net. 

—SHAWN POWERS 


If you're a Linux Journal reader, chances are you're pretty familiar with the 


command line. Don’t let that stop you from heading over to our Web site 


and checking out our daily tech tip videos. Heck, | make the majority of 


them, and | still learn stuff from time to time. Even if you know every tip 


we throw at you, feel free to deflect new users at us. | must warn you, 


however, before you know it, those “n00bs” will end up knowing more tips 


and tricks than you. Don't say we didn’t warn you! 


—SHAWN POWERS -oe Gr : : 
linuxjournal.com/linux-minute 
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Google Voice—One Ring(tone) to Rule Them All 


THERE IS A SCENE in the movie 
Lawnmower Man where every phone 
on the planet rings at the same time. 
Although Google may be the biggest 
technical force on the Internet, | 
don’t think it’s quite to that point. 
With the advent of Google Voice 
(www.google.com/voice), however, 
it’s getting closer. The idea behind GV 
is that a single phone number can be 
used for all your telecommunications. 
Because | finally got an invite to the 
closed beta phase of Google Voice, 

| can explain a bit about it firsthand. 


One Number, Choose Wisely 
When (if?) you get an invite to Google 
Voice, it binds to your Gmail account. 
You get to pick a phone number from 
a giant pool of available numbers. The 
catch is you get to choose only one 
time. Many area codes are available, as 
are many exchanges within them. The 
coolest part about picking a number, 
however, is Google gives you a tool to 
alphanumerically choose. Think (123) 
Coke-Fan might be a nifty number? 
With the tool, you can test for availabili- 
ty. | like the number | got so much, I’m 
almost tempted to post it here—almost. 
Once you pick your number and verify 
at least one telephone, you get to set 
up rules. The rules are amazing. 


No, You Can't Call My Home 
Number 

Based on Caller ID, Google Voice will 
route calls however you desire (Figure 
1). Contacts are placed into groups, and 
rules are set up for specific groups. It's 
possible for certain contacts to ring only 
certain phones, or you can have custom 
greetings for some people or even 
direct some people automatically to 
voice mail. In fact, with “call presenta- 
tion”, you can listen in to voice mail 
while it’s being left. 


The Problem of Calling Others 
One issue with Google Voice is that if you 
call people directly with your cell phone 
(or home phone), they see your actual 
number instead of your Google Voice 
number. Thankfully, Google has a couple 
ways to solve that. If you look up the 
contact on-line, you can click a call but- 


Google voice ~ 
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yy Starred 
History Ss . = aoe 
You can customize how different groups are treated. Have a special greeting for your “Family” or have your 
Spam "Co-workers" ring separate phones. It's all up to you 
Trash 
@ Contacts Group Forwards To Greeting Call 
Presentation 
© Voicemail 
@ sms houston iPhone, Blackberry, Skype Phone, Magic New Custom On 
Jack Greeting 
© Recorded Edit 
BiPeced Cc ki iP Blackb Sk Ph M: New Cust ce) 
oworkers iPhone, Blackberry, Skype Phone, Magic New Custom n 
© Received Edit Jack Greeting 
@ Missed 
Balance 2 Friends iPhone, Blackberry, Skype Phone, Magic New Custom On 
$0 10 Edit Jack Greeting 
Family iPhone, Blackberry, Skype Phone, Magic New Custom On 
Jack Greeting 


Edit 


Manage Groups » 


52009 Google - Terms - Blog - Gooale Home 


Figure 1. Contacts are sorted into groups, which can have drastically different call rules. 


ton that calls both you and the person 
you are trying to reach. You answer your 
phone to hear it ringing on the other 
end. Another option is to call your own 
Google Voice number from one of your 
phones. You get a voice prompt to make 
a call, and then you type in the number 
for the person you are trying to reach. 
Not only is this a way to have others see 
your Google Voice number as the incom- 
ing call, but if you're lucky enough to 
have one of those cell-phone plans that 
allows for a certain number of “free” 
numbers, you can add your Google Voice 
number to your “circle” and never pay 
for minutes again (hopefully, there aren't 
any cell-phone companies reading this). 


Text Messaging and Voice 
Mailing 

One more trick up the sleeve of the big 
Goog is what | refer to as its “Montana 
Magic”. If you get an SMS text message 
to your Google Voice number, it 
automagically is converted to a number 
in the (406) area code. Thankfully, the 
first few letters of the message display 
the contact name, because a random text 
message from an area code in Montana 
might be a bit hard to explain otherwise. 
Once that number is created, however, 
you can use it to call or text directly to 


18 | october 2009 www.linuxjournal.com 


your contact. The number will work only 
if it's called from one of your phones, but 
if you call it, the person on the other end 
sees your Google Voice number instead 
of your actual number. It is rather 
complicated, but brilliant as well. 

Voice transcription, on the other 
hand, isn’t exactly what I'd call brilliant. 
I'm absolutely certain it will get better 
with time. Why am | certain? Because 
it couldn't possibly get worse. | realize 
voice mail and voice-mail transcription 
are both free services, so I'm not really 
complaining, but the quality of tran- 
scription is indescribably horrible. 

If you get a chance to get a 
Google Voice number, or if you were a 
GrandCentral (www.grandcentral.com) 
user that automatically was enrolled 
into the Google Voice beta pool, it really 
is a fun service to play with. | must 
admit, I’m a bit leery about publishing 
the number as my single point of 
contact, because since Google Voice is 
a free service, it could vanish at any 
time. The thought of business cards and 
such being worthless in an instant is a 
bit disheartening. As a way to shield 
your cell-phone number and add some 
phone-spam protection, however, 
Google Voice is quite a nice tool. 

—SHAWN POWERS 
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Lane Becker of Get Satisfaction 
speaksjat Do It With Drupal 2008. 


Learn to harness the power of this flexible open-source social content 
management system and web application framework. 


Do It With Drupal is in its second year and the 2009 event promises 
to be even better than last year. 


e Learn from the world's top Drupal experts 
e Examine and dissect successful Drupal sites 
e Discover new site-building strategies 

e Hear from community-building experts 


¢ Connect with other Drupal professionals 


Registration price includes: 

¢ Admission to all sessions and keynote presentations 

¢ Social and networking events with Drupal's leaders and top talent 
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* Access to entire 2009 Do It With Drupal video archive 


Early registration and hotel room discounts available. 
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RSpec 


A new way of looking at testing. 


Last month, | covered Shoulda, a Ruby gem that 
allows you to test your code using a method 
called behavior-driven development. BDD, as it 

is known, is closely related to test-driven devel- 
opment (TDD), which has become increasingly 
popular during the past few years, particularly 
within the Ruby community. 

In both BDD and TDD, you start to program 
by writing a test that the program should pass, if 
it’s working correctly. Of course, because the 
program hasn’‘t been written yet, the test will 
fail. You then write the smallest amount of code 
possible to ensure that the test passes. When 
that happens, you continue coding by writing 
another test. The fact that your code is tested 
completely gives you the confidence and flexibility 
to “refactor”, moving code around and joining 
it together, without having to worry about intro- 
ducing new, subtle bugs. 

BDD differs from TDD not in its overall 
method, but rather in its approach and semantics. 
BDD concentrates on how things look from the 
outside, rather than from the inside of the code. In 
the case of a Web application, this often means 
looking at things from the user's perspective, or 
if you're a consultant, from the customer's per- 
spective. No longer are you testing the code— 
instead, you are checking that it meets its speci- 
fications. Thus, working with BDD requires that 
you constantly think of yourself as a consumer of 
a particular piece of code, and that you consider 
what it should do at each point, if it is to work 
correctly. | intentionally use the word should 
here, because as you will see, that is an especially 
important word in the RSpec vocabulary, and it 
appears in nearly every test. 

RSpec has become quite popular among Ruby 
programmers in general and Rails programmers in 
particular. It also is closely tied to several other 
high-quality testing technologies, such as Cucumber 
and Celerity, which | will explore in coming months. 
And, although RSpec is not everyone's cup of tea, 
it is popular enough that you should expect to 
encounter it if you do any Ruby development. 
Moreover, it is often good to try something different, 
and RSpec definitely is different, providing a new 
way of looking at testing. 


Installing RSpec 
The home page for RSpec is rspec.info, which 


20 | october 2009 www.linuxjournal.com 


contains instructions for installing RSpec, either on 
its own or as part of a Rails application. I’m looking 
at a simple Rails application this month as an 
example, so you need to install both parts. 

The first requirement is installing two Ruby 
gems, both of which are stored on the popular 
repository for open-source projects, GitHub. You 
can install these gems with: 


sudo gem install rspec rspec-rails -V --source 
http://gems.github.com/ 


(If you already have installed GitHub as a source 
for gem installations, you don't need to specify it 
in this command.) 

Note that if you have older RSpec-related 
gems installed, such as rspec_generator or 
spicycode_rspec_extensions, you probably should 
remove them from your system. Current versions 
of RSpec handle these functions for you, and 
| have encountered problems and conflicts that 
disappeared when | removed those old gems. 

Now that you have RSpec installed, let's create 
a new, simple Rails project. | often like to use an 
address book (and appointment calendar) for my 
examples, so let's create one: 


rails --database=postgresql appointments 


Remember, Rails assumes you have three 
databases for your application, one each for 
the development, test and production environ- 
ments. The database parameters are defined in 
config/database.yml. | assume you are able to 
set these configuration parameters correctly. 
Although you don’t necessarily need a production 
database for the purposes of this column, you 
will need both development and test databases. 

Now you must tell the Rails application to 
include RSpec. There are plugins for RSpec, but 
| generally prefer to use gems when possible. 
Modern versions of Rails allow you to include 
gems in config/environment.rb by adding the 
following two lines: 


config.gem "rspec", :lib => false, :version => ">= 1.2.0" 
config.gem "rspec-rails", :lib => false, :version => ">= 1.2.0" 


With the gems in place, you now can put RSpec 
in place for your Rails application: 


./script/generate rspec 


This creates a spec directory (parallel to the test 
directory, which it effectively replaces). The spec 
directory contains, by default, three files: 


i rcov.opts: setting options for running the 
Ruby coverage tool rcov when run from 
within RSpec. 


rspec.opts: setting options for RSpec itself. 


| spec_helper.rb: a Ruby file containing global 
definitions and configurations for the individual 
specifications, much like test_helper.rb performs 
in Test::Unit. 


With the spec directory in place, you can begin 
to use the special RSpec generators for models, 
controllers and scaffolds. For example, you normally 
would generate a person model with: 


./script/generate model person first_name:text last_name: text 


This still will work, but any automatically gener- 


ated tests will use Test::Unit, installing files into the 
test directory. By contrast, you can use: 


./script/generate rspec_model person first_name:text last_name: text 


This creates the same model file, but also creates 
a skeleton set of RSpec tests. 


/ h Cc 
Let's create a slightly more sophisticated version of 
the person model: 


./script/generate rspec_model person first_name:text \ 
last_name:text email_address:text phone_number: text \ 
sex: text 


This creates a migration, which you can use to 
create the first version of your person model: 


rake db:migrate 


Now, it’s true that you should go into the 
migration file and modify things, such that (for 
example) the person’s name, e-mail address and 
sex are all mandatory. However, let’s ignore that 


-—_ 
-_ 
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Rather, you should imagine the specification 
that a consumer, or the manager, might 
want from a “person” object, and then build 
the object up to adhere to those standards. 


step for now and assume that you want all of 
your validation logic to be at the application layer. 
In such a case, you would want to put some 
validations in the model file. 

Well, you could do it, but that wouldn't be 
very BDD of you, would it? Rather, you should 
imagine the specification that a consumer, or 
the manager, might want from a “person” 
object, and then build the object up to adhere 
to those standards. 

For example, you might want to ensure the pres- 
ence of the first and last names. So, the first file to 
modify is spec/models/person_spec.rb, rather than 
app/models/person.rb. (For reasons | don’t quite 
understand, Test::Unit calls model tests unit tests, 
and RSpec calls them model tests, and the controller 
tests are called functional tests.) If you open that 
file, you'll see a new, bare-bones specification: 


require File.expand_path(File.dirname(__FILE__) + '/../spec_helper') 


describe Person do 
before(:each) do 
@valid_attributes = { 

:first_name => "value for first_name", 
:last_name => "value for last_name", 
:email_address => "value for email_address", 
:phone_number => "value for phone_number" 
sex => "value for sex" 


} 


end 


it "should create a new instance given valid attributes" do 
Person.create! (@valid_attributes) 
end 
end 


You can run your full suite of specs at any time, 
by typing: 


rake spec 


The first line imports anything defined in 
spec_helper, which | mentioned earlier. Next comes 
a describe line; this will be familiar to those of you 
who have looked at Shoulda. The basic idea is 
that someone reading the specification reads the 
argument to “describe” and then reads each of the 
individual specifications that start with “it”. In other 


22 | october 2009 www.linuxjournal.com 


words, this spec file tries to say “Person should 
create a new instance given valid attributes.” 
And, sure enough, it does. 

The before(:each) block tells RSpec what it 
should invoke before each “it” block. This ensures 
that the @valid_attributes instance variable will be 
set to a predictable value before running each spec. 
You then can modify @valid_attributes as necessary 
within each spec, as you will soon see. 

The thing is, you're checking the validity of your 
specification by creating a new instance of Person. 
You can do that, but if the spec fails, you will end 
up with a code backtrace mixed in with your report. 
For this reason, I’m going to change the existing 
spec definition to look like this: 


it "should create a new instance given valid attributes" do 
p = Person.new(@valid_attributes) 
p.should be_valid 
p.save.should_not == false 

end 


Instead of Person.create, you now are invoking 
Person.new, assigning it to the variable p. Let’s 
check p in two different ways, once using should 
and the other using should_not. These methods are 
mixed in by RSpec to the Object class and contain a 
great deal of behind-the-scenes magic to make 
specifications readable, almost as if they were in 
plain English. For example, when you say: 


p.should be valid 


RSpec’s should method looks for a method named 
valid? for that object and checks that the invocation of 
this method returns true. This works for any predicate 
(that is, method that returns true or false). If should or 
should_not is followed by be_XXX, RSpec turns that 
into a method call of XXX? on the object instance. 

So, you can understand what it means to say: 


p.save.should_not == false 


which you equivalently could write in a more 
positive, optimistic way: 


p.save.should == true 


In both cases, you invoke the save method on 
the object and check that its returned value is true. 
You might argue that you don’t need to invoke 
both new and save on your object, but | like to 
make sure the object is valid in both Ruby and the 
database. After all, it could be that you told the 
database to reject null values, but that you allowed 
it using validations in your ActiveRecord definition. 

Now let's move a bit beyond the defaults to set 


some limits on attributes. Presumably, you want people because everyone needs a first name. But when 


in your database to have all of these fields (first name, | run the specs, | get: 

last name, e-mail address, phone number and sex) 

defined. If you were developing in a non-TDD/BDD 1) 

way, you first would set up validations for all of those ‘Person should not be valid without a first name’ FAILED 

and then add some tests. But, here you're trying to expected valid? to return false, got true 

write tests first, thinking from the “outside” how your ./spec/models/person_spec.rb:23: 

object might behave. And indeed, each person should 

have a first name, a last name, an e-mail address and a Finished in 0.038731 seconds 

telephone number. (Strange as it might seem now, 

there was once a time when having an e-mail address 2 examples, 1 failure 

was not expected.) 

So you could, for example, include the following: In other words, the specification failed. But 
that's okay—that's precisely what you want when 

it "should not be valid without a first name" do you're working in BDD fashion. You wrote a test, it 
@valid_attributes.delete[:first_name] failed, and now you can go into the code and mod- 
p = Person.new(@valid_attributes) ify it, so as to ensure that the test passes. Ensuring 
p.should_not be_valid that this current test passes is a simple matter of 
p.save.should == false adding a validation to your ActiveRecord model. 

end Instead of being the empty default: 

In other words, you take @valid_attributes, class Person < ActiveRecord: :Base 

remove the :first_name key from it and then create end 

a new person with the rest of the name-value pairs 

from @valid_attributes. This should not work, you need to make it: 
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These methods are mixed in by RSpec 
to the Object class and contain a great 
deal of behind-the-scenes magic to 
make specifications readable, almost 


as if they were in plain English. 


class Person < ActiveRecord: :Base 
validates _presence_of :first_name 
end 


| save this change, run rake spec again, and 
sure enough, | get: 


Finished in 0.070752 seconds 
2 examples, © failures 


What's next? Now | can move on to the other 
fields, one by one, in order to test them. And 
indeed, this back and forth is precisely the way 
you want to work when you're programming in 
TDD/BDD fashion. You add a spec indicating 
what the object should do, watch the spec fail 
and then add the appropriate line or lines for it 
to work that way. 

You can get a bit fancier than merely checking 
whether attributes exist. RSpec’s should method is 
very powerful, allowing you to check equality (==), 
numeric comparisons (< and >) and regular expres- 
sion matches, among other things. 

When using RSpec on models, to a large degree, 
you can rely on the built-in validations that Rails 
provides. For example, you presumably want the 
sex field to contain either an M or an F. If someone 
enters a value other than that, you should not save 
it to the database. The first step toward such a 
feature is the introduction of a new spec: 


jt "should forbid characters other than M and F" do 
@valid_attributes[:sex] = 'Z' 
p = Person.new(@valid_attributes) 
p.should_not be_valid 
p.save.should == false 
end 


| run rake spec, and find that this test fails. 
Again, that’s to be expected, and now | can modify 
my Person class such that it is more restrictive: 


class Person < ActiveRecord: :Base 
validates_presence_of :first_name 
validates_inclusion_of :sex, :in => %w(M F), :message => 
"Sex must be either M or F" 

end 
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When I run rake spec, | get a failure, but 
not from this latest spec, which passed just fine, 
telling me that Z is illegal. Rather, what fails is 
the first spec, in which @valid_attributes has 
set the key sex to the value for sex. Once 
again, that's fine; the fact that | have moved 
forward in small, incremental steps gives me a 
chance to identify such issues and fix them, 
before things get too out of hand. By modifying 
@valid_attributes such that it uses an M (or an F, 
if you prefer), the specs work. 


Conclusion 

RSpec offers a refreshingly different, but still 
somewhat familiar, approach to issues of testing. 
By thinking in terms of behavior and specifica- 
tions, rather than configuration and internals, 

it becomes easier to create tests. The natural 
“describe”, “it” and “should” terms used in 
RSpec were chosen carefully, and they help turn 
testing into a joint venture among all parties, 
not just programmers. 

Although | covered only built-in RSpec matchers 
(that is, the test that comes after should), it is 
possible, and even encouraged, to create your 
own custom matchers for objects in your project. 

Next month, I'll continue exploring RSpec by 
looking at the ways you can test controllers. This 
raises a number of questions and issues, including 
those having to do with model objects that are 
instantiated while inside a controller. As you will 
see, RSpec’s “mock objects” will make this problem 
much less painful than it otherwise might be.m 


Reuven M. Lerner, a longtime Web/database developer and consultant, is a PhD 
candidate in learning sciences at Northwestern University, studying on-line 
learning communities. He recently returned (with his wife and three children) 
to their home in Modi’in, Israel, after four years in the Chicago area. 


Resources 


The home page for RSpec is rspec.info, and 
it contains installation and configuration 
documentation, as well as pointers to other 
documents. The Pragmatic Programmers 
recently released a book called The RSpec 
Book, written by RSpec maintainer David 
Chelimsky and many others actively involved 
in the RSpec community. If you are interested 
in using RSpec (or its cousin, the BDD tool 
Cucumber), this book is an excellent starting 
point. An RSpec mailing list, which is helpful 
and friendly but fairly high volume, is at 
groups.google.com/group/rspec. 
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Web Server Tricks with 


SRANDOM 


Add some pseudo-randomness to your scripts and user interaction. 


| just migrated onto a newer, bigger server (read 
that as “more expensive”, of course, but because 
my traffic’s justifying it, I'm good with the change). 


To make matters more interesting, | also just bought 


a new laptop (a MacBook Pro), and between the 
two migrations, I’ve been looking through a lot of 
old directories and bumping into all sorts of scripts 
I've written in the past few years. 

The one | thought would be interesting to 
explore here is one | wrote for a pal who was 
involved in a charity and wanted a way to have 
a single URL bounce people 50/50 to one of 
two different Web pages—a sort of mini-load 
balancer, though his application wasn’t quite 
the same. 

The core piece of this is the $RANDOM shell 
variable that's actually kind of magical—each time 
you reference it, you'll find it’s different, even 
though you aren't actually assigning a new value 
to it. For example: 


$ echo $RANDOM 
21960 
$ echo $RANDOM 
19045 
$ echo $RANDOM 
2368 
$ echo $RANDOM 
2425 
$ echo $RANDOM 
10629 


This violates the core user design principles of 
the shell and even the very definition of variables 
(which are supposed to be predictable—if you 
assign the value 37 to it, it still should have that 
value 200 lines and 17 references later). Other 
variables change value based on what you're doing, 
without you actually assigning it a new value, like 
$PWD, but because that’s the present working 
directory, if you move around in the filesystem, it's 
logical that its value would change too. 

The RANDOM value, however, is in a category 
of its own and makes it super easy to add some 
pseudo-randomness to your scripts and user 
interaction (whether it’s truly random is a far more 
complicated—mind-numbingly complex—issue. If 
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you're interested, try Googling “determining the 
randomness of random numbers” to jump down 
that particular rabbit hole. 

In the Bourne Again Shell (bash), RANDOM 
numbers are within the range of 0..MAXINT 
(32,767). To chop it down and make it useful, 
you can simply divide it by the max numeric 
value you seek. 

In other words, if you want a random number 
between 1..10, for example, use the % “remainder” 
function with a call to expr: 


expr $RANDOM % 10 
expr $RANDOM % 10 
expr $RANDOM % 10 
expr $RANDOM % 10 


expr $RANDOM % 10 


DHFAAHMFAOWU HAN HS 


Boiling this down further, how to choose 
between two options randomly now should be 
jumping out of the page at you, dear reader: 


if [ "$(expr $RANDOM % 2 )" -eq "OQ" ] 
conditional expression 


; then 
fi 


If you wanted to be a purist, you also could 
write this with the $(( )) math notation, of course, 
as you'll see a bit later in this column. 

That's enough for us to write the shell script | 
mentioned earlier, the one that randomly switched 
between two possible pages when invoked: 


#!/usr/local/bin/bash 
urll="http://www.bing.com/" 
url2="http://www.google.com/" 


if [ "$ (expr $RANDOM % 2 )" -eq "QO" ] ; then 


echo "Location: $url1"; echo "" 
else 

echo "Location: $url2"; echo "" 
fi 
exit 0 


Can you see what this example script 
does? If you guessed “randomly redirects you 
to either Google or Bing”, you're right! If 
not, well, what the heck? Go back and read 
the code again! 

Now, let's say my friend said “75% of the 
time, | really want to take people to URL1. 
Can you do it, Dave?” 

Here's how that might look: 


if [ "$(expr $RANDOM % 100 )" -1t "75" ] ; then 


(Or, even more clearly as % 4 -1t 3, for 
that matter.) 

If you have more than two choices, you can 
use a case statement that makes uneven alloca- 
tion a bit tricky but otherwise is straightforward: 


case $(( $RANDOM % 4 )) in 


© ) echo $url11; 

1) echo $url2; 

2 ) echo $url3; 

3 ) echo $url4; 
esac 


With this in mind, we could write an n-way 
load-balancing script, so that when people come 
to the home page, they automatically would be 
bounced to one of the rn possible servers. 

The interesting step actually would be to 
round-robin them, based on the server load, 
of course, which could be done by stepping 
through the data using the ruptime command. 
So, given the uptime output of: 


$ ruptime host1 
host1 16:51 up 3+53:17, 3 users, load 0.65 0.68 0.51 


What we really want is to get a list of 
hostnames sorted by how busy those systems 
are, which can be generated by ruptime with 
the -rl flags, as shown here: 


$ ruptime -r -1 

host1 down 16+08:34 

host4 up 10+13:26, 7 users, load 0.07, 0.39, 1.04 
host3 up 14+06:49, 3 users, load 0.10, 0.38, 0.49 
host2 up 1+17:40, 4 users, load 0.18, 0.13, 0.09 


As you can see, the first step is to screen 
out the hosts that aren't actually up at the 


present moment, then grab the first field 
(as it’s sorted by how busy the system is 
at the current moment). 

One approach to this could be to call rup- 
time every time a request comes in and just 
grab the first value. This can be done like so: 


$ ruptime -rl | grep -v down | head -1 | cut -d\ -fl 
host2 


The trouble is that the systems report 
uptime information only approximately 
every minute, and calling ruptime 
dozens or hundreds of times per second 
can end up producing a problem—the 
least-busy system will be swamped. If 
you get a lot of traffic, that’s not going 
to be a manageable solution. 

Here's where we could have our friend 
$RANDOM step back into the picture. 
Instead of always simply picking the 
machine with the lowest load average, 
let's randomly choose one of the three 
least-busy systems. The core snippet 
would look like this: 


getline="$(( ( $RANDOM % 3) + 1 ))" 
targethost="$(ruptime -rl | grep -v down | 


sed -n ${getline}p | cut -d\ -f1)" 


With a bit more code, you could bias 
it so that, say, 50% of the time it would 
pick the least-busy system, 33% of the 
time it would pick the second-least-busy 
system, and 17% of the time it would 
pick the third-least-busy system. As time 
passed and as the load moved around, 
these systems would keep changing, 
and you'd achieve a crude but effective 
load-balancing system. 

Knowing how easily you can select one 
of a number of possible paths randomly in 
a shell script, what else can you imagine 
that would be helpful or just fun? 


Dave Taylor has been involved with UNIX since he first logged 
in to the on-line network in 1980. That means that, yes, he’s 
coming up to the 30-year mark now. You can find him just about 
everywhere on-line, but start here: www.DaveTaylorOnline.com. 
In addition to all his other projects, Dave is now a film critic. 
You can read his reviews at www.DaveOnFilm.com. 
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MICK BAUER 


Brutally Practical Linux 
Desktop Security 


Navigate hostile networks with impunity! 


As | write this month’s column, I’m getting ready 
to attend DEFCON, my all-time favorite information 
security conference and hacker rave party. I'll 
catch up with treasured Known Associates, attend 
cutting-edge technical presentations and drink 
Sam Adams beer two-fisted at Hacker Jeopardy 
(it’s a tough job, but I'm up to it). 

And, at some point, I'll engage in two closely 
related activities: connecting my laptop to the DEFCON 
WLAN (wireless local-area network) to check e-mail, 
hoping fervently that | won't do anything dumb 
enough to expose my passwords or other personal 
information to the thousands of other mischievous 
punks connected to the DEFCON WLAN, and I'll 
have a nervous chuckle or two at the Wall of Sheep, 
a real-time list of WLAN users who have done 
something dumb enough to expose their passwords 
and other personal information to the thousands of 
mischievous punks on the DEFCON WLAN. 

There isn’t necessarily that much shame in 
ending up on the Wall of Sheep. Several years ago 
it happened to none other than world-renowned 
security expert Winn Schwartau. | should mention 
that Winn was a very good sport about it, too—no 
identity theft, no foul, as they say. 

But, that doesn’t mean I’m quite ready to put 
my own reputation on the line without a fight. You 
can bet that before | board the plane for Las Vegas, 
I'm going to lock my laptop down, and when |’m 
there, I'm going to take care of myself like | was 
back home in the hood, on the wrong side of the 
tracks, after dark, with a pork chop hung around 
my neck. Nobody’s going to pwn Mick at DEFCON 
this year without busting out some supernatural 
kung fu. (I hope.) 

So what, you may ask, does any of this have to 
do with those of you who never go to DEFCON and 
generally stick to your friendly local coffee shop 
wireless hotspots and neighborhood cable-modem 
LAN segment? Actually, | think that question pretty 
much answers itself, but I'll spell it out for you: the 
tips and techniques | use to navigate the DEFCON 
WLAN safely with my trusty Linux laptop should 
amply suffice to protect you on whatever public, 
semiprivate or spectacularly hostile networks to 
which you may find yourself having to connect. 

This month’s column is about ruthlessly practical 
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Linux desktop security—what to do to harden your 
system proactively and, even more important, what 
to avoid doing in order to keep it out of harm’s way. 


Overview and Generalities 
Here’s a summary of what I’m about to impart: 


1. Keep fully patched. 


2. Turn off all unnecessary network listeners or 
uninstall them altogether. 


3. Harden your Web browser. 


4. Never do anything important in clear text. 
Actually, do nothing in clear text. 


5. Use VPN software for optimal imperviousness. 
6. Pay attention to SSL certificate errors. 


7. Be careful with Webmail and surf carefully 
in general. 


8. Make backups before you travel. 


Some of those things should be extremely familiar 
to my regular readers, or simple common sense, 
or both. Patching, for example, is both critically 
important and blazingly obviously so. Most network 
attacks begin with a vulnerable piece of software. 
Minimizing the number of known bugs running on 
your system is arguably the single-most important 
thing you can do to secure it. 

I'll leave it to you to use the auto-update tools 
on your Linux distribution of choice, and the same 
goes for making backups, an equally obvious 
(though important) piece of advice. 

At least equally important is minimizing the 
number of software applications that accept 
network connections. If a given application 
either is turned off or has been uninstalled, it 
generally doesn’t matter whether it’s vulnerable 
or not. (Unless, of course, an attacker can 
enable a vulnerable application for purposes of 
privilege escalation, which is one reason you 
should not only disable but also remove unnecessary 


What about Targeted Malware? 


| don’t want you to come away from this with the notion that malware never figures into Linux security or that it never will. In 
settings where you can’t control what software people run or install on their systems or can’t fully enforce automated, timely 


patching, good antivirus software is essential. 


And, | worry quite a bit about targeted malware—that is, hostile code that has been custom created to attack a specific organi- 
zation or individual. That is becoming an increasingly common tool used by organized crime in stealing large quantities of sensi- 
tive data (most typically credit-card numbers and identity data) from specific organizations. Often, the worm or virus will be 
“planted” in the target network by someone with inside access. 


Because a given worm, virus or trojan of this type has been “handcrafted” and never has been released against the general 
public, there's zero likelihood that any antivirus software vendor even will know about it, let alone provide antivirus signatures 
that can detect it. Mainstream, signature-based antivirus software is, therefore, generally useless against targeted malware. For 
this and other reasons, targeted malware is very, very difficult to defend against, even with good patching practices. 


But, this article isn’t about protecting large networks or even about defending yourself from targeted attacks by well-funded 
attackers. It's about protecting yourself from attacks by more or less random strangers you may encounter on the Internet, at 
your local coffee shop's wireless LAN and so forth. And in those contexts, | don’t worry very much about Linux malware. 


applications.) | cover service disabling in 
depth later in this article. 
So far, so obvious. But, what about 


antivirus software? As a matter of fact, Proto 
and by the way I'm waiting for some- tcp 
one to convince me otherwise on this, tcp 
viruses and worms are not a threat | tcp 
take very seriously on Linux. In all my tcp 
years using and experimenting with udp 
Linux, including in university lab settings udp 
and in my own Internet-facing DMZ udp 
networks, | never have had a single udp 
malware infection on any Linux system udp 
| ran or administered. udp 
Is this because there are no Linux udp 
worms or viruses, or because Mick is so udp 
fabulously elite? No, on both counts. udp 


Rather, it’s because I’ve never been lazy 

about keeping current with patches, and 

because I’ve always very stubbornly used plain text 
for all my e-mail. 

Worms exploit vulnerable network applications— 
no vulnerability (or no app), no worm. E-mail viruses 
depend on users executing e-mail attachments or 
on their e-mail software's running scripts embedded 
in HTML-formatted e-mail—no attachment executing 
or script running, no infection. 

I've also been lucky in this regard because 
there are few Linux worms and viruses in the 
wild to begin with. But, even if there were more, 
| would repeat, keeping current with patching 
and using e-mail carefully is more important than 
running antivirus software. 


Listing 1. Network Listeners (Pre-Hardening) 


Recv-Q Send-Q Local Address Foreign Address State 
0 Q@ *:swat ae LISTEN 
0 Q “3ssin esis LISTEN 
0 @ localhost:ipp Bat LISTEN 
0 Om * 31128 chs LISTEN 
0 Q@ <iwazaru:netbios-ns ae 
0 0 *:netbios-ns ma 
0 Q iwazaru:netbios-dgm *:* 

0 Q *:netbios-dgm oe! 
0 @ *:49176 ct 
0 0 *:57500 mae 
0 OR aGpvi2 wus 
0 Q@ *:bootpc ae ey 
0 Q@ *:mdns eee 


Turning Off Network Listeners 

So, assuming you're fully patched already—and 
| assure you | am—let’s get busy disabling net- 
work listeners. The first step in doing this is 

to find them. If | run the command netstat 
--inet -al on my Ubuntu laptop, | see what 
is shown in Listing 1. 

You can see I’m running the Swat front end for 
administering Samba services, the Secure Shell 
deamon, the Internet Printing Protocol and Squid 
(whose default port is TCP 3128). Hmm, you'd 
never guess that | recently wrote articles on Samba 
and Squid, would you? 

Well, those articles are long finished, so right 
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now | don't have any compelling 
reason to keep any of these 
services running, especially when 
| travel. | not only need to shut 


them down, but also disable Proto Recv-Q 
their startup scripts. | could tcp 0 
simply uninstall them, but | udp 0 


might need them later. Still, as 
a general rule, if you can 
uninstall unnecessary software, 
you should. Doing so via your 
preferred package manager is 
simple enough for me to skip 
describing here. 

At the application level, | can 


cleanly. This clobbers the netbios 

nameservice (netbios-ns) and 

netbios datagram (netbios-dgm) 

udp listeners in Listing 1. But, | also need to disable 
the Samba startup scripts and Swat itself. 

Distributions vary in how they handle startup 
scripts for system daemons like these. On SUSE, 
you can use YaST2 or the command insserv. 
On Red Hat, Fedora and CentOS systems, use the 
command chkconf ig. 

Because my system runs Ubuntu, | can use 
either the Services Settings applet (Figure 1) in 
my X Window System's Applications menu or the 
update-rc.d command. Let's start with the Services 
Settings applet, which, by the way, is part of GNOME 
and, therefore, may very well be installed on your 
non-Ubuntu GNOME desktop too. 

Figure 1 shows the Services Settings applet after 
I've already clicked the Unlock button and provided 
my root password. Figure 1 also shows the bottom 
of the list of services running on my system, but 
that’s where some of the juicier items are. | definitely 
want to uncheck the boxes next to Proxy cache 


Services Settings 


Select the services that you wish to activate: 
Printer service (cups) 
gs com 
_—— | 


Allows applications to use printers 


ws o/s Proxy cache service (squid) 


Remote shell server (ssh) 
a Secure shell server 


> Remote backup server (rsync) 


& System communication bus (dbus) 


Web server (apache2) 


@ +=} shares your web pages over the Internet 


| @zuelp 


Unlock | | $€ Close 


Figure 1. GNOME Services Applet 
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bash-$ sudo lsof -i 
COMMAND PID 
use Swat to shut down Samba inetd 29534 


USER FD TYPE 
root 4u IPv4 


Listing 2. Network Listeners (Post-Hardening) 


Active Internet connections (servers and established) 


Send-Q Local Address Foreign Address State 
0 * swat heat LISTEN 
0 *:bootpc sae 


Listing 3. Using lsof to Find Swat’s Process 


iswat 
DEVICE SIZE NODE NAME 
521556 TCP *:swat (LISTEN) 


service (squid), Remote shell server (ssh) and Web 
server (apache2). 

What about Printer service (cups)? I'll disable 
that too, because at DEFCON, it’s highly unlikely I'll 
need to print anything (or even have the opportunity 
to). But, note that as Listing 1 shows, my system is 
listening only for incoming IPP connections on the 
loopback interface (localhost:ipp). It isn’t listening 
for remote connections to this service. 

Me being me, I'll disable it anyhow. A “local” 
attack vector is local only until some other process 
is hijacked by a remote attacker, at which point the 
hijacked process might be used to spawn some 
other process that can attach to the thing having 
the “local” vulnerability. 

Along the same lines—that is, in the interests of 
generalized paranoia—l'Il also disable the following 
in Services Settings (not shown in Figure 1): 


@ Account information resolver (winbind). 

m@ Folder sharing service (samba). 

m@ Multicast DNS service discovery (avahi-daemon). 
@ Network service (xinetd). 


Those are all things I’m sure | can live without in 
an untrusted environment. File sharing in particular, 
in the form of Samba and its winbind service, is 
to be avoided in such settings. Now if | re-run 
my netstat --inet -al command, | see only 
what is shown in Listing 2. 

Not bad! Listing 2 shows that by clicking two 
buttons in the Swat interface and unchecking some 
boxes in the Services Setting applet, I’ve clobbered 
11 out of the 13 network listeners that previously 
had been active on my system. 


Listing 4. Manually Disabling a Startup Script 


bash-$ sudo /etc/init.d/openbsd-inetd stop 
bash-$ sudo update-rc.d -f openbsd-inetd remove 


Removing any system startup links for /etc/init.d/openbsd-inetd ... 


/etc/rc0.d/K20openbsd- inetd 
/etc/rcl.d/K2@openbsd- inetd 
/etc/rc2.d/S20openbsd- inetd 
/etc/rc3.d/S20openbsd- inetd 
/etc/rc4.d/S20openbsd- inetd 
/etc/rc5.d/S20openbsd- inetd 
/etc/rc6.d/K20openbsd- inetd 


Listing 5. Manually Re-enabling a Startup Script 


bash-$ sudo update-rc.d openbsd-inetd start 202345. 
Adding system startup for /etc/init.d/openbsd-inetd ... 
./init.d/openbsd- inetd 
../init.d/openbsd- inetd 
./init.d/openbsd- inetd 
./init.d/openbsd- inetd 
./init.d/openbsd- inetd 
./init.d/openbsd- inetd 
./init.d/openbsd- inetd 


/etc/rcO.d/K2Q0openbsd-inetd -> . 
/etc/rcl.d/K2Q0openbsd-inetd -> 
/etc/rc6.d/K2Qopenbsd-inetd -> . 
/etc/rc2.d/S20openbsd-inetd -> . 
/etc/rc3.d/S20openbsd-inetd -> . 
/etc/rc4.d/S20openbsd-inetd -> . 
/etc/rc5.d/S20openbsd-inetd -> . 


bash-$ sudo /etc/init.d/openbsd-inetd start 
* Starting internet superserver inetd 


But, I’m not done with listeners yet. There still 
are two left. | can't do much about bootpc, which is 
part of the dhcp client daemon that most of us use 
to configure low-level TCP/IP settings automatically 
when we connect to a LAN. Even at DEFCON, I'll 
need dhcpcd (bootpc) active in order to connect to 
the DEFCON WLAN. 

Swat, on the other hand, is fair game to shut 
down, especially considering I've disabled all the 
rest of Samba. But hold on a second, I’ve forgotten 
how! There’s neither a Swat entry anywhere in 
the Services Settings applet nor any applicable 
script in /etc/init.d. Maybe | can figure out the 
name of the actual process listening on the Swat 
port using the Isof (list open files) command, as 
shown in Listing 3. 

Oh, now | remember! Swat is run by inetd, 
which on Ubuntu systems is part of the package 
openbsd-inetd. You may remember my disabling 
xinetd in Services Settings, but openbsd-inetd’s 
startup script has to be disabled manually, the 
old-school Debian way (Listing 4). 

In Listing 4, you can see that | first stopped 
openbsd-inetd via its startup script and then forcibly 


removed the various runlevel- 
links in /etc/rcO.d, etc/rc1.d and 
so forth, via the update-rc.d 
command. | can undo all this 
later, as shown in Listing 5. 

Obviously, | will need to 
make note of the sequence 
number (in this example, 20 for 
both the start and kill links) and 
the runlevels (2-5 for starting 
and 0, 1 and 6 for killing). 

As it happens, the settings for 
openbsd-inetd also are 
Ubuntu’s defaults, so | could 
use the command sudo 
update-rc.d openbsd-inetd 
defaults when re-enabling 
that particular service. 

I've spent the bulk of this 
column shutting down network 
services. Is that all there is to 
system hardening? 

Ordinarily not. If we were 
talking about a server, we'd 
have a lot more work to do: 
configuring individual applica- 
tions for maximum security, 
disabling unnecessary user 
accounts, tightening file permis- 
sions, configuring an integrity 
checker such as tripwire, maybe 
creating a local iptables firewall 
script and so forth. 

But this is my personal laptop, a single-user 
system. Shutting down and disabling unnecessary 
network listeners really is 90% of what | need 
to do to “harden” it. Most of the rest of what 
| need to do concerns how | use this system. 
Before | get to that, however, | need to harden 
one killer application: my Web browser. 


Stop 2050) 116. 


Hardening Firefox 

Firefox's default security settings are surprisingly 
okay. Personally, however, | prefer to disable third- 
party cookies (which admittedly breaks some sites), 
and sometimes | temporarily turn third-party cookies 
back on. | also like to turn off my browsing history 
completely. | don’t need to know where I've been, 
and neither does anybody else. Figure 2 shows 
these privacy settings. 

Under Firefox's Security settings, | certainly want 
to make sure Firefox's default warnings for add-on 
installations, suspected forgeries and other suspected 
hostile sites are intact. | also turn off all password 
caching—the very idea of allowing my browser to 
store passwords is, if you ask me, the way of tears. 
Figure 3 shows these settings. 
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Firefox Preferences 


wes kK EE 


Main Tabs Content Applications Mela Security Advanced 


History 
Keep my history for at least | g0|"| days 


©) Remember what | enter in forms and the search bar 
© Remember what I've downloaded 


Cookies 


Exceptions... | 


@ Accept cookies from sites 
L) Accept third party cookies 


Keep until: | they expire | Show Cookies.., | 


Privale Data 


@ Always clear my private data when | close Fircfox | Settings... | 


1) Ask me before clearing private data | d Clear Now | 


| ME close 


Figure 2. Firefox Privacy Settings 


Firefox Preferences 


K ¢ o 


Ww 
Main Tabs Content Applications Privacy Meetgisal Advanced 


W Warn me when sites try to install add ons | Exceptions... 


@ Tell me if the site I'm visiting is a suspected attack site 
@ Tell me if the site I'm visiting is a suspected forgery 
Passwords 
© Bemernber passwords for sites Exceptions ] 
OC Use a master password | change Master Password. } 
| Saved Passwords... | 
Warning Messages 


Choose which warning messages you want to see while 
browsing the web 


(=) 


@utelr | 


Figure 3. Firefox Security Settings 


Finally, | should mention a couple useful 
Firefox add-ons. | swear by Adblock Plus, which 
enforces a blacklist of known Web advertisement 
sites whose content is frequently streamed to 
various other sites. Blocking those sites effectively 
blocks in-line ads. You can get Adblock Plus by 
searching for “adblock plus” in Firefox’s Get 
add-ons tool, under Tools—Add-ons. 

| realize this breaks various people’s Internet 
revenue streams, but | use Adblock Plus less for 
aesthetic or performance reasons (ad-blocking 
certainly shortens Web site load times) than for 
security. Blocking ads reduces the attack surfaces 
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of the sites you visit and, therefore, reduces your 
chances of being exposed to spyware or other 
hostile content. 

It may be difficult for a given Web hacker to 
compromise nytimes.com directly, but it’s consider- 
ably easier to compromise one or more advertisers 
whose content is loaded in tandem with 
http://www.nytimes.com. Personally, I’m less 
worried about destroying Internet ad revenue 
than | am about protecting my humble browser. 

(Before | forget to mention it, you should 
minimize the number of unfamiliar sites you 
visit in the first place when using an untrusted 
network for the very same reason.) 

Finally, the Firefox add-on Ghostery allows you 
to see what Web bugs (trackers), ad feeds and 
other hidden scripts are active on each Web site 
you visit. For most such scripts, Ghostery can tell 
you from whence it originates and why you should 
or shouldn't worry about it. You can get Ghostery 
at www.ghostery.com. 

Now that Ubuntu and Firefox are hardened 
for DEFCON use, here are some things I'll do when 
actually connected to that wicked DEFCON WLAN 
to minimize my chances of ending up on the Wall 
of Sheep. 


Never Transmit Unencrypted Passwords 
Always, always assume somebody can and will 
eavesdrop on all network traffic. Whether you 
personally can believe or imagine how they'll do 
this or not is unimportant—it’s the attacker's 
imagination and skill that matter here, not yours. 
The only sensible assumption for you to make 
about the network’s integrity is that there isn’t 
any, and that someone can see all traffic going 
to and from your system. Accordingly, you must 
not log on to any remote system through any 
unencrypted protocol. 

Telnet, non-anonymous FTP, IMAP, POP3 and 
any browser-based login involving an http:// URL 
rather than https://, therefore, are all off limits. 
In the modern era, all these applications (remote 
shell, file transfer, e-mail and most Web applica- 
tions) can and should be used in encrypted 
implementations, such as SSH, FTPS or SFTP, 
IMAPS, POP3S and https, at least for logons and 
other sensitive transactions. 


Use VPN 

If your home or corporate network supports it, use 
a strong VPN protocol such as IPsec or SSL-VPN to 
connect back, and do all your Web surfing and 
other Internet business via the home network. Yes, 
this will degrade the performance and speed of 
your Web-surfing experience; however, it will all but 
obliterate risks associated with eavesdropping, 


DNS spoofing, evil twinning and similar attacks 
(although, of course, if your home or corporate 
network is targeted further downstream from 
the hostile LAN you're connected to locally, 
those downstream attacks will apply). 


Care about SSL Certificates 

When using any public, hostile or otherwise 
untrusted network, you must pay careful attention 
to your browser's padlock icon. If there is any 
problem with any certificate being presented by an 
SSL-protected site you've had no issues connecting 
to in the past, you should assume that somebody 
is attempting a man-in-the-middle, proxy or 
imposter Web site attack. 


Be Careful with Webmail and On-line 

Banking 

Gmail, Yahoo, Windows Live (Hotmail) and on-line 
banking sites are all particularly likely for someone 
to attempt to proxy or spoof. If you must visit such 
a site from a hostile LAN, again, watch for any 

certificate weirdness. 


If you have your own Webmail server or have 
access to Webmail from a smaller provider, such as 
a regional ISP, those may be less likely for someone 
to attempt to spoof or proxy than one of the “big 
guys”. For maximum paranoia though, using a 
strong VPN connection really is best. 


Conclusion 

And with that, we're out of space for this month, 
but we're done! If | say so myself, it wasn't a bad 
column's work. My laptop is now hardened for 
DEFCON WLAN use, and you've hopefully learned 
a thing or two about Mick’s brutally pragmatic 
approach to desktop security. We'll see whether 

| end up on the Wall of Sheep this year (if so, 
maybe I'll admit it, and maybe | won't). Good 
luck with your own public LAN adventures! 


Mick Bauer (darth.elmo@wiremonkeys.org) is Network Security Architect for 
one of the US's largest banks. He is the author of the O'Reilly book Linux 
Server Security, 2nd edition (formerly called Building Secure Servers With 
Linu), an occasional presenter at information security conferences and 
composer of the “Network Engineering Polka”. 
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KYLE RANKIN 


Spam: the Ham Hack 


Check out a few simple spam-fighting tweaks to two of my favorite 
open-source programs: mutt and Postfix. 


When you think about it, all spam really is, is 
hacked ham. This is true for both meanings of 
the word. In the food sense, it is composed of 
hacked-up bits of pork that are reassembled to 
resemble (sort of) ham. In the e-mail sense, soam 
is just hacked-up bits of text that are reassembled 
so they somewhat resemble a legitimate e-mail 
you want to read (ham). 

Countless articles talk about the open-source 
tools you can use to reduce the amount of spam 
in your inbox, so I’m not going to write yet 
another article about SpamAssassin, Razor/Pyzor, 
Spam Blackhole Lists (SBLs) or even grey-listing, 
although | recommend looking into those 
countermeasures if you haven't already done 
so. Instead, | assume you already have these 
measures in place, so I’m going to discuss a 
few extra tools that make spam management 
a bit more, well, manageable. 


Virtual Addresses in Postfix 

I'm not a huge fan of Web-based e-mail, although 
for the longest time, | did think it was a great tool 
for spam-catching. | would set up a free Web e-mail 
account, and whenever | bought something on-line, 
| used that e-mail address as a contact. Of course, 


Virtual addresses in Postfix work much 
like aliases work in most mail servers. 


whenever | bought something new, I’d have to go 
into the account first and purge the mountain of 
spam that had accumulated since the last time | 
used the account. The other downside was that | 
still never knew which companies had sold out my 
e-mail address and which ones kept it protected. 

Since then, I’ve found an even better solution 
with virtual addresses in Postfix. Now that | run 
my own mail server, | can set up as many e-mail 
addresses as | want for free and have them all 
land in the same inbox. Not only does this make 
it easier to find all my on-line receipts later, but 
also because of the way | set it up, | easily can 
find out which companies sold me out and block 
only their e-mail messages. 

Virtual addresses in Postfix work much like 
aliases work in most mail servers. It provides you 
a way to set up a large series of To addresses 
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that your mail server will accept and map those 

addresses to one or more real addresses on the 

server, or even forward e-mail to addresses on a 
completely different server. All you have to do is 
set up a new database that defines the mapping 
between virtual and real addresses, and then tell 
Postfix to use it. 

For this example, let's assume | have a mail 
server that already accepts mail for example.net, 
and my personal account is kyle@example.net. 
Whenever | set up a new account, either on a 
social network or an on-line retailer (anything 
that could potentially send me spam), | set up 
a new virtual address named after them and 
the year. Let’s assume | created an account on 
CompanyX’s site, so | could buy a T-shirt and also 
registered a new profile on TweetBookSpace— 
the new hip social-networking-meets-cell- 
phones-meets-LOLcats site. 

First, | would create a regular text file called 
/etc/postfix/virtual that contained the following entries: 


# System accounts that should exist 


kyle@example.net kyle@localhost 
root@example.net kyle@localhost 
# Spam-catching accounts 

companyx2009@example.net kyle@localhost 
tweetbookspace2009@example.net kyle@localhost 


All of the addresses in the left column corre- 
spond to addresses for which Postfix will accept 
mail, and the right column tells Postfix to which 
real account to forward the mail. Instead of an 
@localhost address, | also could forward it to 
some other external e-mail address, or even list 
multiple addresses separated by commas. Once | 
set up the file and whenever | make any changes, 
| need to run the postmap command against it, 
so that it creates the custom database file Postfix 
actually will read: 


$ sudo postmap /etc/postfix/virtual 


Finally, | just need to add some new lines 
to my /etc/postfix/main.cf to define what 
domains | will use for my virtual aliases and 
tell it to use the file | just created. | added 
only one domain here, but if you already have 
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Well, if you didn't already know from my 
prior columns, I’m a big fan of mutt, and 
| didn’t want to be left out of all these 
fancy spam-managing techniques. 


multiple domains defined in your mydestination 
line, move as many as you want managed by 
this file over to the virtual_alias_domains setting: 


virtual_alias_domains = example.net 
virtual_alias_maps = hash:/etc/postfix/virtual 


Then, | can run sudo postfix reload to reload 
my settings. If | start to notice that I’m getting 
spam sent to companyx2009@example.net, all | 
have to do to block that address is comment out 
that line in /etc/postfix/virtual and run postmap 
again. Although it’s not necessary to add the year 
to the e-mail address, I’ve found that helps when 
| periodically go through my old throwaway e-mail 
addresses and comment them out—after all, 
| always can uncomment them the next time | 
want to order something. 


Spam Tips for Mutt Users 

| know plenty of people use whiz-bang graphical 
e-mail programs, and many of them also have fancy 
buttons and icons that flash when e-mail might be 
spam. Well, if you didn’t already know from my 
prior columns, I’m a big fan of mutt, and | didn't 
want to be left out of all these fancy spam-managing 
techniques. Once again, mutt’s powerful customization 
comes to the rescue. 


Colorize Borderline Spam 
Although | do have spam filters set up on my 
personal account, sometimes messages get 
through my defenses. It’s always a delicate 
balancing act when you tweak your spam thresholds, 
so | not only wanted to see how close spam 
that made it through was to the threshold, but 
| also wanted to know if any of my legitimate 
e-mail was close. 

| have SpamAssassin configured so that it 
adds the score to my e-mail headers via the 
custom X-Spam-Status header. Let’s say that my 
spam threshold was a score of 6; | then set up 
two rules: one to color any messages with a 
score of 2 or 3 red and another to color mes- 
sages with a score of 4 or 5 bright red. That 
way, both types of messages would stand out— 
especially the messages right on the tip of my 
threshold. Here are the folder-hook rules | added 
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to my mutt config: 


folder-hook . "color index red default '~h 
= \"X-Spam-Status: .*score=(2|3)\.\"'" 
folder-hook . "color index brightred default '~h 
=»\"X-Spam-Status:.*score=(4/5)\.\"'" 


Now, like many people, | have a special spam 
folder set aside so | can train SpamAssassin. | go 
in there from time to time to look for any false 
positives, so | also wanted to highlight any messages 
that were right above the threshold. The following 
rule colors any messages that have a score of 6, 7 
or 8 magenta: 


folder-hook . "color index magenta default '~h 
w\"X-Spam-Status: .*score=(6|7|/8)\.\"'" 


Quick Macro to Save to the Spam 
Folder 

Now, whenever | go through my inbox and see a 
message with a suspicious Subject line, if | notice it’s 
colored red or bright red, | might not even bother 
to open it. Because | know it's close to the threshold, 
| simply can move it to my spam folder. In mutt, 
you can do this with just a few keystrokes, but of 
course, that doesn't stop me from automating it a 
bit further. After all, why do a few keystrokes when 
| can bind the S key to save to my spam folder 
automatically? All | had to do was add the following 
to my mutt config: 


# make S automatically save spam to the spam folder 
macro index S "simaps://mail.example.net/INBOX.spam" 
macro pager S "simaps://mail.example.net/INBOX.spam" 

Of course, change imaps://mail.example.net/ 
INBOX. spam so that it points to the spam folder on 
your IMAP server, but once you do, you either can 
press S to save an individual message to the spam 
folder or you can tag all of the spam in your inbox 
with the T key, and press ;S to save it all to the 
spam folder at once. 

Sure, it would be great if we never had any 
spam to begin with, but although | can choose 
what canned food | buy at the grocery store, 
| may never fully get rid of spam in my inbox. 
After all, one man’s hacked-up pork by-product 
is another man’s tasty canned-ham substitute. If 
people didn’t order those male-enhancement 
pills, they wouldn’t advertise them. At least with 
a few extra steps, | can make managing spam 
take less time.= 


Kyle Rankin is a Senior Systems Administrator in the San Francisco Bay Area and 
the author of a number of books, including Knoppix Hacks and Ubuntu Hacks for 
O'Reilly Media. He is currently the president of the North Bay Linux Users’ Group. 
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Splunk 


Splunk says that version 4 of its IT search application has hit the streets, offering 
improvements in usability, scalability and performance. Splunk 4 enables users to 
search, analyze, monitor and report on data from any application, server or network 
device in real time to troubleshoot outages, investigate security incidents, meet 
compliance requirements and more—“in minutes instead of hours or days”, says 
the company. Some of the 1,800 enhancements and 50+ new features include 10x 
faster search and 2x faster indexing, custom dashboards for users of any skill level, 
more sophisticated enterprise-level management and the Splunk 4 App Framework 
for creating or leveraging existing apps running on the IT search engine. 


www.splunk.com 
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and external USB/FireWire/e-Sata hard disks and hard disk enclosures. The 
new version 2.9 offers deep disk tests to verify hard disk noise, performance 
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connected to 3ware/AMCC and ARECA RAID controllers and solid-state disk features also can be detected. The Enterprise 
server solution allows monitoring and managing of disk information of remote hosts from a centralized administration console. 
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Making the space for on-line video more interesting is Kaltura Community Edition (KCE), which 

Kaltura dubs “the world’s first and only open-source, self-hosted on-line video platform”. The Ka It uU ra 
freely downloadable KCE allows any site owner or Web developer to integrate customizable video open source video 
and interactive rich-media functionalities, including video management, publishing, uploading, 
importing, syndicating, editing, annotating, remixing, sharing and advertising. Kaltura also claims that KCE breaks the 
“build vs. buy” conundrum and vendor lock-in by allowing publishers and enterprises to build upon and extend an existing 
robust platform to customize fully their own self-hosted solution on their own servers, behind their own firewalls and at no 
cost. The company further offers paid support services. KCE runs on Linux, Mac and Windows and is slated to be available 
on several cloud computing platforms. 
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If you are an administrator who has worked with *nix but is new to virtualization, 
the authorial team of Luke S. Crawford and Chris Takemura has a book for you: 
The Book of Xen from No Starch Press. Xen is a tool that lets administrators run 
many virtual operating systems on one physical server, including Linux, BSD, 
OpenSolaris and Microsoft Windows. In the process, users save money on hardware, 
maintenance and electricity. The book explains everything needed to run Xen, 
covering installation, networking, virtualized storage, and managing guest and 
host operating systems. Beyond the basics, it covers profiling and benchmarks, 
migration, XenSource administration and hardware-assisted virtualization. 


www.nostarch.com 
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David Douglas and Greg Papadopoulos’ 
epee Citizen Engineer (Prentice-Hall) 


A HANDBOOK FOR SOCIALLY RESPONSIBLE ENGINEERING 


Although the engineering discipline has done many wonderful things for civilization, it has at times 
been blind to important social and environmental considerations. In order to foster more humane 
disciplines of engineering, the team of David Douglas and Greg Papadopoulos penned the new book 
Citizen Engineer: A Handbook for Socially Responsible Engineering (Prentice-Hall). Citizen Engineer 
helps engineers of all types to see the full impact of their work beyond design to include ecological, 
intellectual property, business and sociological perspectives. 
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Jedox’s ODBO Driver for Palo 


Led by the kick-butt motto “Excel without the hell”, the company Jedox has announced “the industry's first free 
ODBO [OLE DB for OLAP] driver” as a part of its open-source OLAP product, Palo. Jedox states that the new 
ODBO connectivity allows users to carry out advanced OLAP-based Pivot table queries in Excel without the need 
for expensive licenses for Microsoft SQL Server Analysis Services. Although Pivot tables in Excel are read-only, 
Palo users have the option to write back values from Excel directly to Palo'‘s OLAP cubes. The company calls 
“Excel plus Palo” a solution with all the advantages of a centralized BI solution without the cost and time. 


www.jedox.com 
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Jedox 


Zero9 Chat Engine 


In the pursuit of bringing us closer together comes the new Zero9 Chat Engine, a 
product that enables mobile VAS and telco providers to run image- and video-rich 
chat/dating services via the Web, WAP and SMS. Users can stay in touch with 

Cone ea te friends via their Web browsers, browsing a WAP site or texting with their cell phones. 
The engine's core is Zero9’s Matching Algorithm, which proposes the ideal 
and best-matched friends. A back-office suite controls elements, such as CRM, 
a matching tuner and advanced reporting. The engine is based on the LAMP 
platform and the Zend framework. 
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Corsair’s Extreme Series Solid-State Drives 


The latest offering from Corsair is its Extreme Series X32, X64 and X128 high-performance 
solid-state drives in 32GB, 64GB and 128GB densities, respectively. The firm says 
that the drives offer the highest performance currently available on the market, 
with read speeds of up to 240MB/s and write speeds of up to 170MB/s. Each 
drive in the Extreme Series utilizes the Indilinx Barefoot controller, Samsung MLC 
NAND Flash memory and 64MB of onboard cache. Intended uses are as primary 
drives in desktop and notebooks systems, as well as RAID O configurations in 
high-performance desktops for enthusiasts who want extreme performance. 
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Fresh from the Labs 


htop—Improved, Intuitive 
Version of top 
htop.sourceforge.net 

htop aims to be a better version of the 
long-lived UNIX program, top. Allowing 
you to browse through your system 
processes with the keyboard, htop is 

a much more intuitive way of doing 
things than the traditional and archaic 
UNIX way. The htop Web site gives the 
best summary with this comparison of 
htop and top: 


H In htop, you can scroll the list vertically 
and horizontally to see all processes 
and complete command lines. 


@ In top, you are subject to a delay for 
each unassigned key you press (espe- 
cially annoying when multikey escape 
sequences are triggered by accident). 


@ htop starts faster (top seems to 
collect data for a while before 
displaying anything). 


@ In htop, you don’t need to type the 
process number to kill a process; in 
top, you do. 


@ In htop, you don’t need to type the 
process number or the priority value 
to renice a process; in top, you do. 


m htop supports mouse operation; 
top doesn't. 


@ top is older, hence, more used 
and tested. 


Installation If you're chasing 
binaries, packages are provided in 
either native or third-party form for 
GoboLinux, Debian, Fedora, Red Hat, 
Slackware, Gentoo, ALT Linux, 
OpenSUSE, Mandriva, KateOS and 
Zenwalk. For those going with source, 
head to the Downloads/SVN page, 


htop vastly improves on the old top UNIX tool, 
with an interface that lets you browse through 
your processes with the keyboard. 


| hel: Mop 
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Being able to choose what kind of signal you 
want to send to a process adds versatility. 


grab the latest tarball and extract 
it, or SVN the nightly development 
tree. In terms of requirements for 
compiling the source, the only oddity 
| ran into was that | needed to install 
the development files for ncurses (in 
my case, libncurses5-dev). 

Open a terminal in the new source 
directory (whether from an extracted 
tarball or SVN), and enter the usual: 


$ ./configure 
$ make 


$ sudo make install 


Once the compilation finishes, run 
the program by entering: 


$ htop 


Allowing you to browse through your system 
processes with the keyboard, htop is a much 
more intuitive way of doing things than the 
traditional and archaic UNIX way. 
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Usage Once inside the htop 
screen, you'll see things arranged in 
a way that will make instant sense to 
those used to some Linux mainstays, 
such as xosview, Midnight Commander 
and various system monitors, including, 
of course, top. At the top of the screen 
is a selection of handy system informa- 
tion, such as CPU usage, free memory, 
swap usage and so on. In the center 
of the screen are all of your system’s 
processes waiting to be browsed, and 
in classic Norton/Midnight Commander 
style, the functions of the program are 
sensibly linked to the function keys of 
your keyboard, laid out clearly on the 
bottom of the screen. Take note of 
that in particular, as you'll be using 
those functions the most. 

To get started, you can browse 
through your processes with the arrow 
keys along with Page Up and Page 
Down. If you want to kill or tinker 
with a process, press either K or F9. 

At this point, htop brings up a menu 
of possible signals to send the program, 
with SIGTERM being the default 
choice. Scroll through these if you want 
to play with more-advanced features, 
but for most users, just pressing F9 
and Enter will do the job fine. 

You can kill multiple jobs by 
pressing the spacebar on each one. 
The Search function is mapped to F3, 
allowing you to search for any part 
of text in the name of the process 
you're chasing, which comes in handy 
particularly for overburdened systems 
with too many processes. The SortBy 
function, mapped to F6, also is of great 
help, especially when you want to sort 
between system- and user-owned 
processes. And, the Tree function, 
mapped to F5, is fantastic for seeking 
things like tricky child processes. This 
program is fairly loaded with features, 
so it’s worth checking the htop man 
page and the help screen (assigned to 
F1) for more information. 

Ultimately, htop is a good evolu- 
tionary step from unintuitive, older 
programs like top, and with any luck, 
it will be included by default with 
most distros in the years to come. 
Although the interface still might 
be a little intimidating for novices, 


intermediate or advanced users 
should come to grips with it easily 
(and let's face it, they're the ones 
who'll be using it anyway). 


Open Cubic Player 
stian.cubic.org/project-ocp.php 

Open Cubic Player (OCP) is a text- 
based audio player that runs in 
various incarnations on Linux and 
Windows, and there's even an older 
DOS version. First appearing in late 
1994, the original program was a 
binary-only freeware version called 
Cubic Player (running primarily in 
DOS). It had a reputation for being 
one of the best module players 
around, as it supported a great deal 
of soundcards as well as audio for- 
mats. As Windows grew more popular 
and people demanded GUI-based 
software, popularity and support 
for the program died off, as did the 
project itself. 


Open Cubic Player gives wonderful visualizations 
rendered in real time with pure ASCII. 


Thankfully, browsing for audio files is an easy 
affair, and there are many advanced features 
that major GUI players don’t even have. 


Eventually, the source code was 
opened up to the public in the hope 
that someone would find it useful, 
and in late 2003, developer Stian 
Sebastian Skjelstad started playing 


around with it, attempting to get the 
source to compile and run under 
Linux. After a great deal of tinkering 
around, Stian eventually got some- 
thing working, and today, it’s available 
in beta form. And quite frankly, it's 
a little ripper of a player! 
Installation Binary packages are 
provided for Debian and Ubuntu, as 
well as some specific information for 
installation on other systems, but if 
you're not using the basic .deb pack- 
ages, you might as well install OCP 


from source. Grab the latest tarball 
from the Web site, extract it, and 
open a terminal in the new folder. 
As for strange requirements with the 
source, | had to grab the development 
files for both ogg and vorbis, which 
were liboggz1-dev and libvorbis-dev, 
respectively. Being a wacky console 
program, you probably need the 
ncurses development libraries too, 
but | already had those on my system 
after compiling htop (see above). 
When it comes to compiling the 
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source, documentation is sorely lacking, 
but thankfully, compilation is a simple 
case of the usual: 


$ ./configure 
$ make 
$ sudo make install 


When the compilation is over, run 


Projects ata 


UnixTree 

www.unixtree.org 

Much like Midnight Commander 
captured the feel and essence 

of the popular DOS application, 
Norton Commander, UnixTree 
models itself closely on the once 
popular XTreeGold. XTreeGold was 
the introduction that many first PC 
users had with DOS, which, much 
like Norton Commander, had a 
semi-GUI interface to ease the 
transition into a tricky environment. 
Although this may not strike an 
instant chord with command-line 
purists or full-blown X users, | know 
a number of people for whom 
XTreeGold was their primary 
interface day to day, and hopefully, 
UnixTree will ease their transition 
into the UNIX shell in the same 
way XTreeGold did for DOS. I've 
had a chance to use it, and I’m 
quite impressed, especially as 
certain essential UNIX commands 
are assigned to single keystrokes 
to speed up your daily command- 
line usage. 


Stateline 


UnixTree 


the program by entering: 
$ ocp 
Usage Although I’m still coming 


to grips with the basic controls, playing 
singular files is a simple affair, as is 


exploring the program's many functions. 


When you enter the OCP screen, your 


Glance 


Gnake 

lightless.org/gnake 

Anyone who has a mobile phone will 
know the time-old classic game, Snake. 
| still play the popular X game Gnibbles 
from time to time, and I've always had 
a soft spot for the style of gameplay. 
Gnake brings that gameplay to the 
console in a rather simplified form, but 
still, it's damn hard. Compilation is easy; 
simply grab the tarball, enter make and 
then ./gnake. I’m not sure whether 
the levels progress (I haven't passed 
stage one), but some of the options 
that can be altered include the play- 
ground size, speed, number of apples, 
growing length and the ability to add 
computer-controlled snakes. 


Console Commander 
concom.sourceforge.net 

Console Commander brings you a 
selection of information and system 
tools under an easy-to-navigate group 
of menus that should be of serious 
comfort to anyone not familiar with 
the Linux shell. Some of the clever 
features group together the sorts of 
information I've always had to dig 
through large GUI programs to reach, 
like Kinfocenter. Information, such as 
CPU type, free memory, partition 
usage, distro info and so on, is usually 
a pain to hunt down individually. 
Combined with tools that automate 
tasks like package and repository 
upgrades, how-tos and many more 
features, this is a handy program 
indeed. Although there’s nothing tech- 
nically amazing happening here, it has 
no pretenses about what it is and will 
save time for many users. | love it. 


first encounter is with the file browser, 
where you can select your songs, 
append them to a playlist and so on. 
I'm not too sure how to operate the 
playlist functions confidently enough 
to explain them (you can work out the 
contents of the manual yourself), but 
playing a single file is easy. Simply 
search for the file you want with the 
up and down arrow keys. Entering 
directories or playing a file is done 
by pressing the Enter key. 

When a track is playing, this whole 
project comes to life, and the point 
becomes clear—you instantly have full 
visualizations of your music along with 
neat power-level indicators and all 
manner of tinkering functions. This is 
designed for control freaks—seriously. 
On screen is a load of information, 
right down to file size, frequency 
and format information, and so on. 
However, it's the functions that are the 
meat of the program. You can alter 
the panning, balance, speed, pitch, 
amplification and more. You even can 
turn on a surround function—not bad 
for a text-based player! 

These functions are mostly spread 
over the function keys, but the 
coolest feature (although admittedly 
a little gimmicky) is actually pausing a 
song. Press P, and your song winds 
down and dies like someone has just 
pulled the plug on an old reel-to-reel 
player. Unpausing winds it back up to 
life again. It's really cool and adds 
genuine charm to the player. 

Working your way around this 
program is initially unintuitive, and the 
documentation feels as if it’s written 
more for other programmers than new 
users, but the charm of this program is 
unavoidable. The beautiful spectrum 
analyzer patterns rendered in real-time 
ASCII are enough to bring a tear to any 
geek’s eye, and the advanced controls 
one expects only of complex, resource- 
intensive GUI applications will entrench 
this player firmly in the heart of many a 
technophile. Awesome stuff—if you 
can work your way around it!m 


John Knight is a 25-year-old, drumming- and climbing- 
obsessed maniac from the world’s most isolated city—Perth, 
Western Australia. He can usually be found either buried in an 
Audacity screen or thrashing a kick-drum beyond recognition. 


Brewing something fresh, innovative or mind-bending? Send e-mail to newprojects@linuxjournal.com. 
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Celebrating 15 years of Linux Journal, 
we ve brought together every article 
ever published in the world's #1 
Linux magazine and packaged it 
In one convenient CD. 


release date: february 2003 


Aes, 
JOURNAL 
ARCHIVE 1994-2008 


Includes issues 1-176 of Linux Journal 


With nearly 4,000 articles written by industry experts on everything from cool projects, desktop how-tos, 
security, embedded systems, networking, virtualization, multimedia, system administration and 


programming tricks and techniques—this unique collection is a must-have for every Linux enthusiast. 


Get your NEW Linux Journal Archive CD today featuring 
all issues from 1994 through 2008. Just $34.95. 


www.linuxjournal.com/archivecd 


coreboot 


AT YOUR SERVICE! 


Don’t let your PC’s closed-source BIOS stop you 
from doing what you want with your hardware. 


ANTON BORISOV 


s people started to build large computing clusters from ordinary PCs, the shortcomings of existing PC BlOSes 
for certain tasks became more obvious. Like any other computer, on occasion, a cluster’s nodes need to be 
rebooted; however, most of the original PC BlIOSes halted on boot if no keyboard was attached. Obviously, 
adding a keyboard and monitor to every node in a large cluster is not feasible. These days, this particular 
problem has been fixed by most PC BlOSes. They contain an option that tells the system to continue booting even if 
there is no keyboard. Other problems persist, of course, such as how to reboot and adjust the BIOS settings remotely. 
One of the first people to try to fix these problems was Ron Minnich from Advanced Computing Lab, LANL, who in 
1999 started the open-source BIOS project named LinuxBlOS. In 2008, the project’s name was changed to coreboot. 
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Development Phases 
The project has had three different phases: LinuxBIOS v1, 
LinuxBlOS v2 (or coreboot v2) and coreboot v3. 

The first phase, LinuxBIOS v1, began in 1998-1999, and 
it became a “finished” product in 2000. At this point in the 
project, the BIOS consisted of some simple hardware initialization 
code, and the rest was a stripped-down version of the Linux 
kernel to do the real initialization. Because the Linux kernel 
does a lot of hardware initialization during its boot process 
(tests memory, sets up interrupts and so forth), it seemed like 
a reasonable choice to use the existing Linux kernel—hence 
the name LinuxBlOS. 

The main problem in LinuxBlOS v1 was writing correct code 
so that the multitude of available motherboards were initialized 
properly. The code was far from “universal”. Among other 
things, each motherboard had its own unique memory 
initialization sequence, and most of the motherboard 
initialization code was written in assembly. 

In the second phase, the developers took a new approach. 
They left the assembly code to enter protected mode 
untouched, but they rewrote everything else in C. There was a 
bit of a problem though. Normally, code 
generated by a C compiler assumes a stack 
is available, but because memory has not 
yet been initialized, there is no stack 


In Practice 

My lab contains a VIA EPIA-M Il for test purposes. It was 
manufactured a few years ago, but it's supported by coreboot. 
Let’s take a look at how it is easy to replace its closed-source, 
proprietary BIOS with the open-sourced coreboot. 

Because the EPIA-M Il is not yet supported by coreboot v3, 
I'm going to cover installing v2 for this example. First, make 
sure you have GCC, binutils, Python, bash, pciutils-devel and 
subversion installed. Now, check out coreboot v2 code from 
the repository: 


$ svn co svn://coreboot.org/repos/trunk/coreboot-v2 
Next, fetch a payload: 
$ svn co svn://coreboot.org/filo 
| decided to use FILO, which is almost the same as LILO, 
but it uses no BIOS calls. You also may use GRUB2 if you like; 


it’s completely compatible with coreboot. 
You also need a special library named libpayload, because 


Listing 1. Configuration Process for libpayload 


available. To get around this problem, Eric ss 

Biederman created a new C compiler called * Libpayload Configuration 

ROMCC. As you may have guessed, * 

ROMCC generates machine code from C * Architecture Options 

that uses only CPU registers—meaning * 

machine code that needs no stack and, Multiboot header support (MULTIBOOT) [Y / n] 
therefore, no initialized RAM! Plus, the * 

CPU's cache is used as RAM. This technique * Standard Libraries 


is now known as CAR (Cache-As-RAM). * 
Although, LinuxBlOS v2 fixed some of 
the original design’s problems, others 
remained. For instance, in order to add or * 
remove a “payload”—the code that is 
actually responsible for loading the operating * 
system—you had to recompile LinuxBlOS. 
Around 2006, the developers refined 
their approach again. This, the current 
phase, is coreboot v3. Coreboot v3 uses 
the Kconfig facility to set all configuration 
settings—the same way you recompile a 
“normal” Linux kernel. The coreboot image 
is now an archive file that allows modules 
to be added to and/or removed from an 
image more easily. Also of note in coreboot 


Enable C library support (LIBC) [Y / n] 
Enable tinycurses support (TINYCURSES) [Y / n] 


* Console Options 


See output on the serial port console (SERIAL_CONSOLE) [Y / n] 
I/O base for the serial port (SERIAL_IOBASE) [0x3f8] 
Override the serial console baud rate (SERIAL_SET_SPEED) [N / y] 
Use plain ASCII characters for ACS (SERIAL_ACS FALLBACK) [N / y /?] 
See output on a video console (VIDEO_CONSOLE) [Y / n] 
VGA video console driver (VGA_VIDEO_CONSOLE) [Y / n] 
Geode LX video console driver (GEODELX_VIDEO_CONSOLE) [N / y] 
Allow input from a PC keyboard (PC_KEYBOARD) [Y / n] 
English (US) keyboard layout (PC_KEYBOARD_LAYOUT_US) [Y / n] 
German keyboard layout (PC_KEYBOARD_LAYOUT_DE) [N / y] 


v3 is the dropping of ROMCC—all code 
is compiled with gcc. Due to marketing 
reasons, the project's name was changed 
from LinuxBlOS to coreboot. 

LinuxBIOS v1 supported 64 motherboards, 
and LinuxBlOS v2 supported about 120. 
The current version, coreboot v3, is still 
young, and at the time of this writing, it 
supports only 16 different motherboards. 


* 


* Drivers 

x 

Support for PCI devices (PCI) [Y / n] 

Support for reading / writing NVRAM bytes (NVRAM) [Y / n] 

Extended RTC ports are @x74/0x75 (RTC_PORT_EXTENDED VIA) [N / y /?] 
Support for PC speaker (SPEAKER) [Y / n] 

USB Support (USB) [N / y] 
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FEATURE Coreboot at Your Service! 


Listing 2. Configuration Process for FILO 


bi ReiserFS (FSYS_REISERFS) [Y / n] 
* FILO Configuration XFS (FSYS_XFS) [N / y] 
‘ 1809660 filesystem (FSYS 1S09660) [Y / n] 
* Interface Options El Torito bootable CDROMs (ELTORITO) [Y / n /?] 
i Compressed RAM filesystem (CRAMFS) (FSYS_CRAMFS) [N / y] 
Use GRUB like interface (USE GRUB) [Y / n /?] Squash filesystem (FSYS_SQUASHFS) [N / y] 
Command line prompt (PROMPT) [filo] ‘ 
GRUB menu.lst filename (MENULST_FILE) [hda3:/boot/grub/menu. 1st] * Loaders 
Timeout for loading menu.lst (MENULST TIMEOUT) [0] iy 
Use MD5 passwords in menu.lst? (USE_MD5 PASSWORDS) [Y / n /?] Standard Linux Loader (LINUX_LOADER) [Y / n /? 
SS Windows CE Loader (WINCE_LOADER) [N / y /?] 
* Drivers Artec Loader (ARTEC_BOOT) [N / y /?] 
* * 
IDE DISK support (IDE_DISK) [Y / n /?] * Debugging & Experimental 
IDE disk poll delay (IDE_DISK_POLL_DELAY) [0] x 
Extra delay for SATA (SLOW_SATA) [N / y /?] Enable experimental features (EXPERIMENTAL) [N / y /? 
PCMCIA CF (Epia) support (PCMCIA_CF) [N / y /?] DEBUG_ALL (DEBUG_ALL) [N / y] 
new USB Stack (USB_NEW DISK) [Y / n /?] DEBUG_ELFBOOT (DEBUG_ELFBOOT) [N / y] 
USB Stack (obsolete?) (USB DISK) [N / y /?] DEBUG_ELFNOTE (DEBUG_ELFNOTE) [N / y] 
NAND Flash support (FLASH DISK) [N / y /?] DEBUG_SEGMENT (DEBUG_SEGMENT) [N / y] 
PCI support (SUPPORT PCI) [Y / n] DEBUG_SYS_INFO (DEBUG_SYS_ INFO) [N / y] 
Scan all PCI busses (PCI_BRUTE_SCAN) [N / y /?] DEBUG_BLOCKDEV (DEBUG _BLOCKDEV) [N / y] 
Sound Support (SUPPORT SOUND) [N / y] DEBUG_VFS (DEBUG_VFS) [N / y] 
* DEBUG_FSYS_EXT2FS (DEBUG_FSYS_EXT2FS) [N / y] 
* Filesystems DEBUG_PCI (DEBUG_PCI) [N / y] 
‘ DEBUG_LINUXLOAD (DEBUG_LINUXLOAD) [N / y] 
EXT2 filesystem (FSYS_EXT2FS) [Y / n] DEBUG_IDE (DEBUG_IDE) [N / y] 
FAT (MSDOS) filesystem (FSYS FAT) [Y / n] DEBUG_ELTORITO (DEBUG_ELTORITO) [N / y] 
JFS (PSYS_JFS) (N / y] Developer Tools (DEVELOPER TOOLS) [Y / n /?] 


Minix filesystem (FSYS MINIX) [N / y] 


FILO depends on it. Check it out, and then run make, which running, you can come back and switch off the options you 
first will run through the configuration: don’t need, which will reduce the size of the coreboot image. 
Notice the following line near the top of Listing 2: 
$ svn co svn://coreboot.org/repos/trunk/payloads/libpayload 
$ cd Libpayload GRUB menu.lst filename (MENULST FILE) [hda3:/boot/grub/menu. 1st] 
$ make 
My test EPIA-M Il system has OpenSUSE 11.0 installed and 
Listing 1 shows the output from the configuration process. uses the GRUB bootloader. | chose to include GRUB's interface 
Simply press Enter for all options. The value chosen 
is the default, which is the capitalized value in square 


brackets [...] if it’s a yes/no option; otherwise, it’s the Listing 3. If your distribution works with LILO, you can switch off the GRUB 
value in brackets. interface in FILO. 
Once the configuration parameters are set, run 
make again to compile the library: * 
* FILO Configuration 
$ cd ../filo * 
$ make * Interface Options 
* 
Now you can set the options for FILO. Again, Use GRUB like interface (USE_GRUB) [Y / n /?] n 
simply press Enter for all the prompts and accept Autoboot a command line after timeout? (USE_AUTOBOOT) [Y / n] 
the defaults. Kernel filename and parameters (AUTOBOOT FILE) 
Obviously, not all of the above options actually > [hdal:/boot/vmlinuz root=/dev/hda3 console=tty0 
are needed. For instance, you don’t need XFS, =console=ttyS0, 115200] 
JFS or Minix support if your system boots off an Time in seconds before booting (AUTOBOOT_DELAY) [2] 


ext2/ext3 partition. Once you've gotten everything 
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support inside FILO, and this is the place to specify the location 
of GRUB’s menu file. If you don’t plan to use the GRUB 
interface (for instance, if your Linux distribution uses LILO 
for booting), you need to specify the correct line to load the 
kernel and initrd, as shown in Listing 3. 

After setting the FILO’s configuration parameters, compile 
FILO by running make again. The compiled loader is placed 
here: filo/build/filo.elf. 

At this point, you've prepared the payload. Now, you need 
to generate a coreboot image. First, let's take a look at the 
config file that is used during the coreboot build (Listing 4): 


$ cd coreboot-v2/targets/via/epia-m 
$ vi Config.1b 


Lines 1 and 2 define the board and board manufacturer 
that makes the board we’re targeting. Lines 3-5 set the 
logging level. Higher values give you more information, and 
logging information comes out on a serial (RS-232) port. 

Line 6 specifies the size of the Flash (ROM) memory chip 
on your board. 

Line 7 indicates that coreboot may access CMOS memory 


Listing 4. Configuration Parameters for Coreboot v2 


R 


target epia-m 
2 mainboard via/epia-m 


3 option MAXIMUM_CONSOLE_LOGLEVEL=8 
4 option DEFAULT_CONSOLE_LOGLEVEL=8 
5 option CONFIG_CONSOLE_SERIAL8250=1 


6 option ROM_SIZE=256*1024 


7 option HAVE_OPTION TABLE=1 
8 option CONFIG ROM PAYLOAD=1 
9 option HAVE_FALLBACK_BOOT=1 
10 option CONFIG COMPRESSED PAYLOAD_NRV2B=1 


11 option FALLBACK_SIZE=131072 


12 option _RAMBASE=0x00004000 


13 romimage "normal" 

14 option USE_FALLBACK_IMAGE=0 

15 option ROM_IMAGE_SIZE=64*1024 

16 option COREBOOT_EXTRA_VERSION=".0-Normal" 
17 payload $ (HOME)/filo/build/filo.elf 

18 end 


19 romimage "fallback" 

20 option USE_FALLBACK_IMAGE=1 

21 option ROM_IMAGE_SIZE=60*1024 

22 option COREBOOT_EXTRA_VERSION=".0-Fallback" 
23 payload $ (HOME)/filo/build/filo.elf 

24 end 


25 buildrom ./coreboot.rom ROM_SIZE "normal" "fallback" 


for getting any parameters—in particular, the boot sequence. 

Line 8 specifies that the boot image (payload) is located in 
ROM. In some situations you will want to load the payload via 
a serial port. For those cases, use this: 


CONFIG_SERIAL_PAYLOAD=1 


Line 9 sets the strategy used to start coreboot. For example, 
if the checksum from CMOS-memory is not valid, instead of 
loading the “normal” part, coreboot must start the backup 
part—that is, “fallback”. 

Line 10 specifies the compression method (NRV2B). 
Because Flash chip sizes are somewhat limited, you can (or 
may have to) use a compressed payload. Instead of NRV2B, 
you can use LZMA—a more-advanced method: 


CONFIG_COMPRESSED_PAYLOAD_LZMA=1 


Line 11 specifies the size of the backup (fallback) part: 
128kB, half the size of the Flash chip. 

Line 12 indicates where exactly in RAM the compressed 
coreboot will be placed upon power-up. 

Lines 13-18 and 19-24 are almost identical except for 
name and ID. Here you define the “normal” and “fallback” 
parts. If coreboot can’t start the “normal” part for some 
reason, it will start the reserved, “fallback” part instead. 

The last line specifies how the build tool must combine 
both parts into a single file. See Resources for more informa- 
tion on all of these options. 

That's all for the configuration; now compile coreboot for 
the EPIA-M: 


$ cd coreboot-v2/ 

$ ./buildtarget via/epia-m 
$ cd via/epia-m/epia-m/ 

$ make 


The coreboot image is ready. The next step is writing it into 
the Flash chip. To do this, you need a special tool, flashrom, 
which comes with the coreboot sources: 


Figure 1. BIOS Savior is a must-have tool. 
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FEATURE Coreboot at Your Service! 


$ cd coreboot-v2/util/flashrom/ 
$ make 


Before proceeding, take note, if problems occur when 
writing to the Flash or if you've configured coreboot improperly 
(such as forgetting to include a payload), you can brick your 
hardware. Therefore, it's highly recommended that you have 
a way to restore your BIOS, such as by using BIOS Savior from 
IOSS (Figure 1). 

To write to the Flash chip, execute the following command: 


# ./flashrom -w ~/coreboot-v2/targets/via/epia-m/epia-m/coreboot.rom 
Then, verify that Flash has been written correctly: 
# ./flashrom -v ~/coreboot-v2/targets/via/epia-m/epia-m/coreboot.rom 


In order to see boot messages with OpenSUSE 11.0, | first 
need to modify my GRUB configuration to set the serial line to 
a speed of 115200 (Listing 5). Now, when | start my EPIA-M, 
| will be able to see coreboot's output in minicom. 


Listing 5. Modifications added to GRUB’s menu.lst in order to 
redirect output to serial port COM1. 


serial --unit=0 --speed=115200 
terminal serial 


default 0 
timeout 8 
gfxmenu (hd0,2)/boot/message 


title openSUSE 11.0 - 2.6.25.5-1.1 
root (hd@,2) 
kernel /boot/vmlinuz-2.6.25.5-1.1-default 
= root=/dev/sda3 resume=/dev/sda5 
“>splash=silent showopts vga=0x317 
=console=ttyS0,115200n8 
initrd /boot/initrd-2.6.25.5-1.1-default 


anthony Maplog's shew Wo.4 Konsole ae x 


Figure 2. You can control the booting process via minicom. 


48 | october 2009 www.linuxjournal.com 


Figure 3. The operating system writes to serial port. 


You now should be ready to reboot, so shut down the 
EPIA-M, connect a null-modem serial cable, and run minicom: 


# minicom -o -8 ttyUSB 


Next, restart the EPIA-M, and minicom should show you 
a GRUB-like boot menu (Figure 2). As the system boots, the 
operating systems’ boot messages also appear in minicom 
(Figure 3). 


QEMU and Coreboot 

Both coreboot v2 and v3 allow you to use the QEMU emulator for 

doing all the above steps without worrying about bricking your 

hardware. Using an emulator also is handy when you want to 

develop a new payload or re-implement some feature of coreboot. 
The steps for configuring and compiling coreboot for an 

emulated system are similar to those for the EPIA-M: 


$ cd coreboot-v2/ 

$ ./buildtarget emulation/qemu-x86 
$ cd emulation/qemu-x86/qemu-x86/ 
$§ make 


Before you can run the emulator, you need to to have a 
copy of a Video ROM/BIOS patched for use with QEMU, such 
as the one for Cirrus Logic card (www.coreboot.org/ 


G) mc -Wéicoreboot viibuild ShellNo. 2 Konsole 
Session Edit View Wookmarks Settings tlelp 


CPU Information 


Vendor: Intel 
Processur: 

Family: 6 

Model: 3 

Stepping: 4 

CPU Speed: 1597 Mhz 


Features: 

fpu de pse toc mcr pac mce cx apic sep pge cmov pat 
mmx I'xsr sse ssez 

Intel Extended Flags: 

Sse. 
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Figure 4. QEMU is a powerful tool that greatly helps to develop coreboot. 


images/0/0d/Vgabios-cirrus.zip). Download the file, unzip it 
in the current directory and then run QEMU: 


$ qemu -L . -hda /dev/zero 


The latest version of coreboot (v3) doesn’t include support 
for many motherboards yet, but anyone can evaluate coreboot 
v3 with an emulated system (Figure 4). 


Coreboot v3 
So, what are the major differences that distinguish coreboot 
v3 from the previous releases? First, the configuration mechanism 
is greatly revised. In previous releases, you had to edit 
configuration files manually. Now, you configure coreboot just 
as you configure the Linux kernel—using make menuconfig or 
make xconfig. Second, the coreboot image itself is nothing 
but a LAR archive. LAR is a coreboot-specific archiver. It allows 
you to add, edit and delete payloads in a single step; there’s 
no need to recompile the entire image. Third, the process for 
producing the code has been simplified and is much more 
elegant than before. Fourth, the use of ROMCC has been 
dropped, and all C code now is compiled with gcc. Fifth, there 
is a growing community and improved documentation on the 
Web site, as well as feedback from some silicon companies. 
So, let's take a look at coreboot v3. Get the source code 
from the repository and configure it: 


$ svn co svn://coreboot.org/repos/trunk/coreboot-v3 
$ make menuconfig 


$ make 
Once this completes, the coreboot image is ready and can 


Listing 6. Output of LAR Archiver (Coreboot Image Contents) 


normal/option_table (932 bytes @ 0x50); 
oadaddress 0x0 entry 0x0 
normal/initram/segmentO (420 bytes @ @x450) ; 
oadaddress 0x0 entry Ox0x40 
normal/stage2/segmentO (194,780 bytes, 
zeroes compressed to 1 bytes @ Qx650); 
oadaddress Ox0xd7b0 entry 0x0x2000 
normal/stage2/segment1 (34560 bytes, 
lzma compressed to 18320 bytes @ Ox6bQ) ; 
oadaddress 0x0x2000 entry 0x0x2000 
normal/stage2/segment2 (6076 bytes, 
lzma compressed to 356 bytes @ 0x4e90) ; 
oadaddress OxOxbff4 entry 0x0x2000 
normal/payload/segment® (183,984 bytes, 


zeroes compressed to 1 bytes @ 0x5050) ; 
oadaddress 0x0x318e0 entry 0x0x19000 
normal/payload/segment1 (100,552 bytes, 

lzma compressed to 78196 bytes @ Ox50b0) ; 
oadaddress 0x0x19000 entry 0x0x19000 
bootblock (20480 bytes @ 0x3b000) 


Total size = 119314B 116KB (0x1d212) 


ASA 
COMPUTERS 


Want your business to be more productive? 


The ASA Servers powered by the Intel Xeon Processor provide the 
quality and dependability to keep up with your growing business. 


Hardware Systems for the Open Source 
Community - Since 1989. 
(Linux, FreeBSD, NetBSD, OpenBSD, Solaris, MS, etc. 


1U Server - ASA1401i 

- 1TB Storage Installed. Max - 3TB. 

- Intel Dual core 5030 CPU (Qty-1), Max-2 CPUs 

- 1GB 667MGZ FBDIMMs Installed. 

, - Supports 16GB FBDIMM. 
fms - 4X250GB htswap SATA.II Drives Installed. 
STN ee = / - 4 port SATA-II RAID controller. 
£2 ? - 2X10/100/1000 LAN onboard. 


2U Server - ASA2121i 

- 4TB Storage Installed. Max - 12 TB. 

- Intel Dual core 5050 CPU. 

- 1GB 667 MGZ FBDIMMs Installed. 

- Supports 16GB FBDIMM. 

- 16 port SATA-II RAID controller. 

- 16X250GB htswap SATA-II Drives Installed. 
~2X10/100/1000 LAN onboard. 

-800w Red PS. 


3U Server - ASA3161i 
- 4TB Storage Installed, Max - 12TB. 
- Intel Dual core 5050 CPU. 
- 1GB 667MGZ F BDIMMs Installed. 
’ _ Supports 16GB FBDIMM. 
- 16 port SATA-II RAID controller. 
- 16X250GB htswap SATA-II Drives Installed. 
- 2X10/100/1000 LAN onboard. 
- §00w Red PS. 


5U Server - ASA5241i 

- 6TB Storage Installed. Max - 18TB. 

- Intel Dual core 5050 CPU, 

- 4GB 667MGZ FBDIMMs Installed. 

- Supports 16GB FBDIMM. 

- 24X250GB htswap SATA-II Drives Installed. 
- 24 port SATA-II RAID. CARD/BBU. 

- 2X10/100/1000 LAN onboard. 

-930w Red PS. 


8U Server - ASA8421i 


- 10TB Storage Installed. Max - 30TB. 

- Intel Dual core 5050 CPU. 

- Quantity 42 Installed. 

- 1GB 667MGZ FBDIMMs. 

- Supports 32GB FBDIMM. 

- 40X250GB htswap SATA-II Drives Installed. 
~ 2X12 Port SATA-II Multilane RAID controller. 
- 1X16 Port SATA-II Multilane RAID controller. 
- 2X10/100/1000 LAN onboard. 


- 1300 W Red Ps. 


All systems installed and tested with user's choice of Linux 
distribution (free). ASA Collocation—$75 per month 
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be found in build/coreboot.rom. To view the contents of 
coreboot.rom, you can use the LAR archiver (Listing 6): 


$ build/util/lar/lar -1 build/coreboot.rom 


As you can see, coreboot.rom is really just an archive file, 
so it can be “disassembled” and “reassembled”. Compiling 
rom scratch is not required. 


Sophisticated Boot 

Most contemporary, proprietary BIOSes contain a BIOS setup 

program, where you can configure different settings, ranging 
rom RAM parameters to the boot strategy. Currently, there is 
no such thing available for coreboot, but to illustrate the 

lexibility of coreboot, let's examine bayou. 

Bayou was developed by AMD and contributed to the 
coreboot project last year. Bayou is a payload that itself is a 
container for further payloads, thereby allowing coreboot to 
choose among payloads at boot time via a menu. For instance, 
you could include a memtest payload, a FILO payload and 
even a tint payload (tint is a Tetris clone). As an example, let's 
build bayou with a tint payload. 

To build tint, get the source and patch it: 


$ wget http://ftp.debian.org/debian/pool/main/t/tint/ 
=tint_0.03b.tar.gz 

$ tar xfvz tint_0.03b.tar.gz 

$ cd tint-0.03b 

$ svn export svn://coreboot.org/repos/trunk/payloads/ 
wexternal/tint/libpayload_tint.patch 

$ patch -pl < libpayload_tint.patch 

$ make 


Then, get the bayou payload: 


$ svn co svn://coreboot.org/repos/trunk/payloads/bayou 
$ cd bayou 


Edit the bayou configuration file (bayou.xml), and add the 


Listing 7. Configuration for Bayou Payload 


<BayouConfig> 
<global> 
<timeout>5</timeout> 
</global> 
<payloads> 
<payload type="chooser" flags="default"> 
<file>payloads/filo.elf</file> 
</payload> 
<payload type="chooser"> 
<file>payloads/coreinfo.elf</file> 
</payload> 
<payload type="chooser" flags="default"> 
<file>payloads/tint.elf</file> 
</payload> 
</payloads> 
</BayouConfig> 
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required payloads (Listing 7). 
The config file is fairly straightforward. The default payload 
starts after five seconds, and if nothing is chosen, the default 
is FILO. The other options are tint or coreinfo (Figure 4 shows 
it running in QEMU). 
Next, make a directory for the payloads, put them into it, 
and then run make: 


$ 1s -1 payloads 


-~PWXP-XVr-Xx 


= PWXP =X =X 


-PWXP-xXP-Xx 


-PWXP-XP-Xx 


1 ab users 
1 ab users 
1 ab users 
1 ab users 


47004 2009-01-03 11:59 
71440 2009-01-09 21:35 
49298 2009-01-10 09:40 
74334 2009-01-10 19:22 


coreinfo.elf 
Tile elt 
PainiteLe 
seabios.elf 


$§ make 


Now, run image in QEMU, and you can play Tetris from the 
BIOS (Figure 5). 


QEMU — 


Your Icvc STATISTICS 
Full lines 


= 


Sum 


Score ratio 
Eff icicney 


Figure 5. Take a rest—Tetris running from the BIOS. 


Conclusion 

Coreboot allows you to replace your motherboard'’s proprietary 
BIOS with a free and open-source BIOS. Coreboot already can 
boot Windows XP and Windows Vista as well as FreeBSD, 
Plan9 and, of course, Linux. Operating systems can be started 
from local disks, from a network connection or even from a 
serial port. Although it’s not yet feature-complete, coreboot 
provides a base for building more flexible BlOSes.m 


Anton Borisov lives and works in Russia. Always fond of low-level programming, he has devoted 
his PhD work to the economic analysis of the advantages and ROI of custom-made firmware. 


Resources 


Coreboot: www.coreboot.org 


Coreboot Options: 
www.coreboot.org/Coreboot_Options 


IOSS: www.ioss.com.tw 


QEMU: bellard.org/qemu 
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FPGA 


PROGRAMMING 
WITH LINUX 


Short of opening your own chip fab, 
you cant get much closer to the 
metal than FPGA programming. 


ree software licenses and operating systems like GNU/Linux make 
it possible to learn programming and customize state-of-the-art 
software in countless ways. Hacking software, however, isn't the 
last frontier anymore. What if you could hack integrated circuits 
directly—that is, tell a chip to connect its internal transistors to 


create exactly the custom, real-time digital hardware you want? This 


is precisely what you can do with Field Programmable Gate Arrays 
(FPGAs). In this article, | explain how to do it with nothing else but 
your Linux computer and an inexpensive development board. 

This isn’t the first time Linux Journal has covered FPGAs (see Resources), but these 
devices have made huge progress since those articles were written. Plus, prices for powerful 
development boards have come down dramatically. Today, you can do really cool things 
with a $200-$300 FPGA-based board, a typical personal computer and one square foot of 
desk space. Finally, the FPGA community is now big enough and stable enough to make life 
much easier for beginners. The major FRGA makers offer text or video tutorials and forums 
where even newbies can find support. Above all, Web sites like Opencores.org codevelop 
and release Linux-compatible FPGAs and boards like the EUS 100LX with the same spirit 
(and licenses) as free software. In summary, the barrier to entry is much lower now than 
it was even just a couple years ago, making FPGA design both a cool hobby and an 


affordable and interesting addition to the offerings of even high schools. 


MARCO FIORETTI 
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An FPGA Is Not a Microprocessor 

A digital integrated circuit (IC) is a chip that deals only with 
binary digits—meaning signals that can assume only one of 
two states: 0 or 1, high or low voltage and so on. An FPGA 
is an IC consisting of one array of digital logic gates. These 
basic circuits, made up of a few transistors each, instantiate 
either a flip-flop or lookup tables capable of implementing any 
boolean function of up to four binary signals. The magic of 
FPGAs is that the connections among the logic gates (the 
actual circuit you need) are made at power-up by reading the 
configuration instructions written into a bit file. Changing 
the file changes the function of the FPGA. 

As flexible as they are, microprocessors are always and only 
microprocessors: single-purpose hardware that is capable only 
of executing (relatively slowly) instructions in one machine 
language. An FPGA, instead, becomes whatever hardware you 
need. It can morph into a microprocessor, a game console, a 
real-time IP switch or encryption device, an antitheft server or 
anything else you can imagine. The only limits are that your 
circuit cannot require more transistors or external pins than 
those physically present on the chip, and it can’t go faster 
than the intrinsic propagation delay from gate to gate. 

That said, modern FPGAs are powerful enough to let you 
squeeze several Linux-compatible microprocessors, like the 
Nios, the PowerPC or the Microblaze, inside them and still 
have lots of room for your own custom circuits. In many 
cases, you can load certified CPU designs from libraries and 
place them on silicon with simple commands, creating very 
flexible, complete systems inside just one chip. 

The major manufacturers of FPGAs and other programmable 
ICs are Altera and Xilinx, followed by Lattice and Atmel. 
Although the example in this article uses Xilinx products, 
the general procedure is the same with all vendors, and all 
of them have similar boards. In all cases, the design soft- 
ware is closed-source and often expensive, but it is possible 
to download either free trial versions valid for one or two 
months, or free versions with reduced functionality but free 
upgrades and no expiration date. 

The current way to design FPGAs is to write a behavioral 
model in a Hardware Description Language (HDL), like Verilog 
or VHDL, which supports concurrency and synchronous circuits. 
Concurrency allows you to create fully parallel, independent 
processes, each describing how to update some variables 
continuously. Synchronous circuits, instead, are those made 
of flip-flops that change their state only on the edge of some 
clock signal. 

After the design has been written and verified with an HDL 
simulator, a compiler creates a list of all the logic gates and 
the wires (nets) that must connect them to reproduce the 
functionality of the HDL model. After this logic synthesis, 
layout programs read the netlist and several constraints files 
to find out which logic gates inside the FPGAs must be used 
and which physical, internal wires must connect them to 
each other. The end result is the bit file that the FPGA reads 
at power-up. 

The official Xilinx design suite is called ISE Foundation 
(www.xilinx.com/ise), and the reduced functionality version 
is called Webpack. Both programs run on Windows, Red Hat 
Enterprise and SUSE Linux Enterprise (32- or 64-bit). Other 
Linux distributions may work too, but there is no guarantee. 


ISE has a graphical installer where you must accept 
the software license and enter the key you got after a free 
registration on the Xilinx Web site. After it’s finished, you'll 
find a script called settings32.sh or settings64.sh in the 
installation directory—that’s the one you have to source 
to add the Xilinx software to your path. After this, type 
ise at the prompt to launch the Project Navigator (Figure 
1). This is a front end to a bunch of specialized programs, 
one for each design phase. You also can run most of these 
back-end utilities from the command line. The Navigator 
includes a Tcl prompt and, if you select Project~Generate 
Tcl Script, it will save all the commands you entered through 
the GUI as a Tcl script. 
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Figure 1. Xilinx Design Suite 


Other ISE components, like the simulator, the FPGA 
Editor and ChipScope, have graphical interfaces. You'd use 
the FPGA Editor to place and connect single gates manually 
when the software fails to do it according to your specs. 
ChipScope is like a software oscilloscope with a USB 
probe. During synthesis, you can add special circuits to 
your design that will buffer the internal signals you want 


Figure 2. Spartan-3AN Starter Kit 


www.linuxjournal.com october 2009 | 53 


FEATURE FPGA Programming with Linux 


- SXiinxe 
SPARTAN® . 
XC3S700AN™ 
FGG484AGQ072 
0177524 


4CES 


ne 


FATWA 


=> - 
66 

ON 
oé $4 


Figure 4. Stereo Mini-Jack, Serial M, Serial F, Keyboard, VGA, Power 
(Ethernet/USB on Other Edge) 


to see and send them over a USB cable to the ChipScope 
software for display. We'll see the ISE HDL simulator at 
work in a moment. 

The board | got from Xilinx for this article is the Spartan-3AN 
Starter Kit (Figure 2), based on the Spartan XC3S700AN FPGA 
(Figure 3), which contains about 700K system gates. Around 
it there are several memory chips, a 50MHz onboard clock, a 
connector for an external clock and several extra components, 
from D/A and A/D converters to generic I/O pins, assorted 


LEDs, sliders and push buttons and, finally, a two-line LCD 
display. The ports (Figure 4), are enough to make a full-custom 
Linux PC out of this board: 10/100 Ethernet/PHY, USB, key- 
board, VGA, serial and stereo mini-jack for PWM audio. A 
universal power adapter and USB cable are included, as are 
four different bit files that demonstrate the capabilities of the 
FPGA. The corresponding design files are freely downloadable 
from the Xilinx Web site. 
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Figure 5. Starter Kit LCD Screen 
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(Xilinx macro) 


out_port 


dnt_and_new_chars 


Figure 6. Simplified Circuit Diagram of the DNA Reader 


In order to show you what it's like to design custom digital 
hardware and how FPGA development software works, I've 
modified one of the demo circuits loaded into the Starter Kit, 
the DNA reader by Xilinx Senior Engineer Ken Chapman. 
Spartan FPGAs have a unique ID number, called DNA. 
The DNA reader displays the intro string “DNA Reader by Ken 
Chapman”, and then this number is displayed on the LCD 
screen (Figure 5), working as shown in Figure 6. An Xilinx 
hardware macro called dna_port reads the DNA ID from the 
silicon. A PicoBlaze processor first displays the intro string, 
then gets the DNA ID from dna_port and finally sends it, one 
character at a time, to the LCD interface through the Icd_d 
data bus. The PicoBlaze code is stored into the dna_ctrl ROM. 
My modification consists of a small extra circuit that 
overwrites the default intro string on the fly with one saying 
“M Fioretti Linux Journal”. Be warned that this is a hack made 
only for demo purposes. In the real world, if you actually 
needed to change that string, it would make much more sense 
to rewrite the PicoBlaze assembly code. Because this is an arti- 
cle about HDL design in FPGAs, however, | went for a solution 
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based on easy-to-read HDL code whose effect is easily visible circuit, the reading_dna module. You must declare all internal 
in one picture. registers and wires before using them (lines 17-23). HDLs sup- 
My extra circuit is shown in red in Figure 6: a counter and port hierarchy; you can instantiate other modules by declaring 
decoder that detect when the PicoBlaze is driving the LCD them and connecting all their ports to the right signals (lines 
data port (Icd_d) and send different characters to it. The VHDL 26-49). Line 53 shows a first example of a synchronous 
source code corresponding to this extra hardware is shown in process. Depending on the value of the cnt_ops counter, 
Listing 1, which is not the complete, working VHDL file | used, whenever there is a positive edge of the clock (line 56) and 
but only an excerpt meant to give you an idea of how HDL the processor sets the signals write_strobe and port_id(6) high, 
coding works. the Icd_output_data register loads the character from the 
Lines 1-12 define input and output ports of the top-level processor or the one from my extra logic (lines 62-68). The 


Listing 1. VHDL Source Code 


1 entity reading dna is 46 program_rom: dna_ctrl 

2 Port ( led : out std_logic_vector(7 downto 0); 47 port map( address => address, 

3 cd_d : inout std_logic_vector(7 downto 0); 48 instruction => instruction, 

4 cd_rs : out std_logic; 49 clk => clk); 

5 cd_rw : out std_logic; 50 

6 cde : out std_logic; 51 kcpsm3_reset <= '0'; 

7 j2_30 : out std_logic; 5D 

8 j2_26 : out std_logic; 53 output_ports: process(clk) 

9 j2_22 : out std_logic; 54 begin 

10 j2_14 : out std_logic; 55 

11 clk : in std_logic); 56 if clk'event and clk='1' then 

12 end reading_dna; 57 if write_strobe='1' then 

13 = 58 

14 architecture Behavioral of reading_dna is 59 -- 8-bit LCD data output address 40 hex. 
15 os 60 

16 61 if port_id(6)='1' then 

dy, -- start extra signals for LJ demo 62 -- lcd_output_data <= out_port; 

18 signal cd_e_copy : std_logic; 63 --extra code for LJ demo 
19 signal cd_e_del_l : std_logic; 64 if ((cnt_ops >= 8 and cnt_ops <= 17) or 
20 signal cd_e_del_2 : std_logic; 65 (cnt_ops >= 19 and cnt_ops <= 32)) then 
21 signal current_character : std_logic_vector(7 downto 0); 66 lcd_output_data <= current_character; 
22 signal  cnt_ops : integer range @ to 49999999 := Q; 67 else 

23 -- end extra signals for LJ demo 68 1cd_output_data <= out_port; 

24 begin 69 end if; --end extra code for LJ demo 
25 70 end if; 

26 device_dna: dna_port 71 

27 port map( din => dna_din, 72 end if; 

28 read => dna_read, 2B 

29 shift => dna_shift, 74 end if; 

30 dout => dna_dout, 75 

3! clk => dna_clk); 76 end process output_ports; 

32 77 

33 processor: kcpsm3 2B -- LCD interface 

34 port map( address => address, 79 

35 instruction => instruction, 80 cnt_and_new_chars: process(clk) 

36 port_id => port_id, 81 begin 

37 write_strobe => write_strobe, 82 if clk'event and clk='1' then 

38 out_port => out_port, 83 

39 read_strobe => read_strobe, 84 if port_id(5)='1' and write_strobe='1' then 
40 in_port => in_port, 85 Icd_e_copy <= out_port(0); 

41 interrupt => interrupt, 86 end if; 

42 interrupt_ack => interrupt_ack, 87 

43 reset => kcpsm3_reset, 88 lcd_e_del_1 <= lcd_e copy; 

44 clk => clk); 89 Icd_e_del_2 <= lcd_e_del_1; 

45 90 
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Figure 7. Simulation of Listing 1 
91 if (1cd_e_copy ='1' and lcd_e_del_1='0') then -- posedge 
92 if cnt_ops=49999999 then -- inc counter 
93 cnt_ops <= 0; 
94 else 
98 cnt_ops <= cnt_ops + 1; Proven technology. Proven reliability. 
96 end if; -- if cnt_ops=49999999 . . 
e ee ie eee Ce ce ite you can’t afford to take chances with your business 
a= ile: ata or productivity, rely on a GS-1245 Server powered by 
38 the Intel® Xeon® Processors. 
99 if (lcd_e_del_l ='1' and Icd_e_del_2='0') then -- posedge 
100 case cnt_ops is -- character generator 
101 when 8 => current_character <= "01001101"; -- M 
102 when 9 => current_character <= "00100000"; -- space 
103 when 10 => current_character <= "01000110"; -- F 
104 when 11 => current_character <= "01101001"; -- i 
105 when 12 => current_character <= "01101111"; -- o 
106 when 13 => current_character <= "01110010"; -- r 
fey wen | T4l=0 current character’ <=s roll @ii@t"= = 6 Ideal for high density clustering in standard 1U form factor. Upto 16 
108 when 15 => current_character <= "01110100"; -- t Cores for high CPU needs. Easy to configure failover nodes. 
109 when 16 => current_character <= "01110100"; -- t Featu res: 
110 when 17 => current_character <= "01101001"; -- i -1U rack-optimized chassis (1.75in.) 
11 - Up to 2 Quad Core Intel® Xeon® Woodcrest per 
112 when 19 => current_character <= "91001100"; -- L Node with 1600 MHz system bus 
113 when 20 => current character <= "1101001"; -- i - Up to 16 Woodcrest Cores Per 1U rackspace 
114 when 21 => current_character <= "01101110"; -- n = Up to 64GB DDR2.667 & 533 SDRAM Fully 
115 when 22 => current_character <= "01110101"; -- u Buffered DIMM (FB-DIMM) Per Node 
116 when 23 => current_character <= "01111000"; -- x - Dual-port Gigabit Ethernet Per Node 
117 when 24 => current_character <= "00100000"; -- space - 2 SATA Removable HDD Per Node 
118 when 25 => current_character <= "01001010"; -- J a. (x8) mobs ues Ret Node 
119 when 26 => current_character <= "01101111"; -- o 
120 when 27 => current_character <= "01110101"; -- u 
121 when 28 => current_character <= "01110010"; -- r 
122 when 29 => current_character <= "01101110"; -- n 
123 when 30 => current_character <= "01100001"; -- a 
124 when 31 => current_character <= "01101100"; -- 1 
125 when 32 => current_character <= "00100000"; -- space 
126 
127 when others => current_character <= "00100000"; -- space 
128 
2g end case; 
130 end if; -- end (1cd_e del_1 ='1' and 1cd_e del_2='0') 
mi 780 Montague Express. # 604 
3 San Jose, CA 95131 
133 end if; -- clk'event and clk='1' Www.genstor.com 
134 end process cnt_and_new_chars; i : eeepcre . 
Phone: 1-877-25 SERVER or 1-408-383-0120 
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Figure 8. Final Output Report 


Figure 9. Program Running on FPGA Hardware 


cnt_and_new_chars process starting at line 80 does the real 
work. First, it samples the LCD enable signal to count (line 91) 
the write accesses to the LCD. One cycle after a write occurs, 
working with the new counter value (line 99), the process cal- 
culates the next current_character that should be displayed. If 
you look at lines 101-125 you'll see that, instead of the DNA 
number, the display should show the ASCII string “M Fioretti 


The procedure to transform 
this really simple HDL model 
into properly connected gates 
on silicon is equally simple. 


Linux Journal”. A quick simulation (Figure 7) proves that the 
new process sends those characters to the display at the right 
times—that is, when the Icd_rs signal is high (low would 
indicate LCD configuration commands). 

The procedure to transform this really simple HDL 
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model into properly connected gates on silicon is equally 
simple. Double-click, one at a time, the icons in the left- 
center pane of the Project Navigator shown in Figure 1: 
synthesize, Implement Design, Generate Programming File 
and Configure Target Device. If you clicked directly on the 
last one, ISE would do all the previous steps in the right 
order anyway, but doing it in steps is a better way to learn. 
Eventually, you'll get the bit file and a final report like the 
one shown in Figure 8, showing how much silicon was 
used. Remember, what we just did is actual hardware— 
that is, transistors directly connected to do, in real time, 
what we ordered them to do. All that remains to make it 
actually happen is to load the bit file in the FPGA. Figure 9 
shows the result. 


Conclusion 

Due to space constraints, | have given only a very limited view 
of the FPGA design flow. Pushing FPGAs to the limits requires 
lots of skill and experience. | have said nothing about floor 
planning, optimization or simulation strategies, nor have | 
gone into how to run Linux inside FPGAs. All these are 
excellent topics for future articles. 

My goal with this article was simply to show that it is very 
easy to start learning these skills, and that there already is a 
strong community to help you. Students, for example, might 
consider whether FPGAs are what they need to become 
the next Linus or Steve Jobs. In my opinion, any high school 
already teaching programming should add FPGA to its courses. 
If yours is already doing it, please let me know. 
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Linux KVM 


as a Learning Tool 


Low-level system programming is a difficult task, 
but with Linux KVM, it’s a whole lot easier. 


DUILIO JAVIER PROTTI 


ow-level system programming is a difficult task, and 

acquiring expertise in the areas of interrupt handling 

and memory segmentation/paging can be a time- 

consuming and frustrating process if you're working 
right down on the metal. An alternative choice is to use a 
virtual machine or the Linux KVM module to create and run 
your own mini-kernels from scratch quickly. 


The KVM Module 

The KVM (Kernel-based Virtual Machine) module turns a Linux 
host into a VMM (Virtual Machine Monitor), and it has been 
included in the mainline Linux kernel since version 2.6.20. A 
VMM allows multiple operating systems to run concurrently 
on a computer. These guest operating systems execute on 
the real (physical) processor, but the VMM (or hypervisor) 
retains selective control over certain real system resources, 
such as the physical memory and the I/O capabilities. 

When a guest tries to perform an action on a controlled 
resource, the VMM takes control from the guest and executes 
the action in a fashion that keeps it from interfering with 
other guest operating systems. As far as the guest knows, it 
thinks it is running on a platform with no VMM—+that is, it has 
the illusion of running on a real machine. For example, the 
guest can do memory paging and segmentation and interrupt 
manipulation without interfering with the same mechanisms 
within other guest operating systems or within the VMM itself. 

A normal Linux process has two modes of execution: 
kernel mode and user mode. KVM adds a third one: guest 
mode (Figure 1). When a guest process is executing non-l/O 
guest code, it will run in guest mode or perhaps better-named 
guest-user mode. This is the “user” mode shown inside the 
“Guest mode” box in Figure 1. In kernel mode (guest-kernel), 
the process handles exits from guest-user mode due to I/O or 
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other special instructions. This is the “kernel” mode shown 
inside the “Guest mode” box in Figure 1. In user mode, the 
process performs I/O on behalf of the guest. This is the “I/O 
Ops” box shown within the normal “User mode” box in 
Figure 1. For more on how KVM itself operates, see the KVM 
Web site and the many on-line articles about it. 


User mode Guest mode 


Linux Process (VM 
launcher) 


VM Exits 


Kernel mode 


Figure 1. KVM Modes of Execution 


The examples presented here require a recent Linux kernel 
with the KVM module installed and the LibKVM library to 
interact with the module from userspace. You can install the 
corresponding package(s) from your favorite distribution or 
compile the KVM source package (from SourceForge) to create 
both the module and LibKVM library. Note that the KVM 
module works only on platforms with hardware support for 
virtualization; most newer Intel and AMD 64-bit-capable 
processors have this support. 

The rest of this article shows how to build a series of 
guest-mode programs (kernels) as well as a user-mode 
program to emulate their I/O (a virtual machine launcher). 


What Gets Virtualized? mov $0xOa,%al 


The basic components of contemporaneous computer outb al ,$Oxf1 // output value 0x@a to I/0 port Oxf1 
machines are memory, one or more CPUs and one or more I/O 

devices. Therefore, a virtual computer machine should have the guest will exit from guest mode, and the configured outb() 
these three kinds of components. Linux KVM has the ability to callback function is called in user mode (with values Oxf1 and 


handle the virtual machine’s memory and CPUs (with hardware 
help). The third ingredient, I/O, currently is left to the programmer 


and has to be handled in a custom way. Listing 1. LibKVM Methods Used for Our Launcher 

For instance, the KVM distribution comes with qemu-kvm, 
a modified QEMU program that builds virtual machines using kvm_context_t kvm init(struct kvm_callbacks  *callbacks, 
LibKVM and emulates various I/O devices, such as a VGA card, void *opaque) ; 
PS/2 mouse and keyboard and an IDE disk. We are not going 
to use gemu-kvm here, but rather we will code a virtual machine int kvm_create(kvm_context_t kvm, 
launcher from scratch to keep our first examples simple and to unsigned long  phys_mem_bytes, 
learn how a program like qemu-kvm does its work. void **phys_mem) ; 
How to Create a Virtual Machine Launcher int kvm_create_vcpu(kvm_context_t kvm, 
The KVM module exposes a character device (/dev/kvm) for int slot); 
interaction with userspace. For simplicity, we won't access this 
device directly but instead through LibKVM (API defined in void *kvm_create phys mem(kvm_context_t kvm, 
libkvm.h). Use the methods shown in Listing 1 to build the unsigned long phys _start, 
virtual machine launcher (code based on Avi Kivity’s test driver unsigned long len, 
program included in the KVM sources). int log, 

To start, create a KVM context with kvm_init(). The first int writable) ; 
argument is a kvm_callbacks structure to specify the handlers 
to be called when I/O or some system-sensitive instructions are int kvm_run(kvm_context_t — kvm, 
executed inside the virtual machine—for example, when the int vcpu) ; 


guest executes something like this: 


@Avecuanics 


visit us at www.siliconmechanics.com 


or call us toll free at 866-352-1173 inside” 


Carlos knows a good thing when he sees it, but as one of the Powerful. 
Silicon Mechanics Quality Assurance Experts, that’s his job. When it Intelligent. 
comes to the new Storform iServ R515, Carlos sees more than just 

the care and attention to detail that go into the production of your 

custom-configured, versatile storage server. 


The new R515 from Silicon Mechanics is a 4U server with 24 hot-swap SAS/SATA drives, 
optional integrated SAS controller, 6 PCI expansion slots, and redundant power. It features 
the new Intel® Xeon® processor 5500 series with Intel Turbo Boost Technology that delivers 
additional performance automatically when you need it. It also offers smart features like 
energy-saving DDR3 memory. With 2 internal SATA drive bays, you may want to think about 
incorporating Intel X25-E Extreme Solid State Drives for fast, reliable, energy-efficient OS 
drives. And Carlos will make sure that the servers we build and ship to you are exactly what 
you ordered. 


When you partner with Silicon Mechanics, you get more than cost-effective, 
flexible, high-capacity storage—you get an expert like Carlos. 


Ss Expert included. 


FEATURE Linux KVM as a Learning Tool 


Listing 2. I/O Callbacks (used in launcher.c) 


static int my_inb(void *opaque, intl6_t addr, uint8_t *data) 
{ puts ("inb"); return 0; } 


static int my_inw(void *opaque, uint16_t addr, uint16_t *data) 


a 


puts ("inw"); return 0; } 


static int my_inl(void *opaque, uintl6_t addr, uint32_t *data) 


oS 


puts ("inl"); return 0; } 


static int my_outb(void *opaque, uint1l6_t addr, uint8_t data) 
{ puts ("outb"); return 0; } 


static int my_outw(void *opaque, uintl6_t addr, uint16_t data) 
{ puts ("outw"); return 0; } 


static int my_outl (void *opaque, uint16_t addr, uint32_t data) 


{ puts ("outl"); return 0; } 


static int my_pre_kvm_run(void *opaque, int vcpu) 
{ retinn ia. } 


. and similar for my_mmio_read, my_mmio_write, 
my_debug, my_halt, my_shutdown, my_io window, 
my_try_push_interrupts, my_try_push_nmi, 
my_post_kvm_run, and my_tpr_access 


static struct kvm_callbacks my_callbacks = { 


.inb = my_inb, 

.inw = my_inw, 

inl = my_inl, 

.outb = my_outb, 

.outw = my_outw, 

-outl = my_outl, 

.fmio_read = my_mmio_read, 
.mmio_write = my_mmio_write, 

. debug = my_debug, 

shalt = my_halt, 

.10_window = my_i0_window, 
.try_push_interrupts = my_try_push_interrupts, 
.try_push_nmi = my_try_push_nmi, // added in kvm-77 
.post_kvm_run = my_post_kvm_run, 
.pre_kvm_run = my_pre_kvm_run, 

. tpr_access = my_tpr_access 


OxOa for its second and third parameters, respectively). 

Initially, use dummy callbacks. Create and reference them 
in a variable called my_callbacks, as shown in Listing 2. Most 
field names are self-explanatory, but for a brief description 
of each of them, refer to the comments in the structure 
definition in libkvm.h. 

To create the virtual machine itself, use kvm_create(), 
whose second argument is the amount of RAM in bytes 
desired for it, and the third argument is the address of a 
location that will in turn contain the address of the beginning 
of the memory space reserved for the virtual machine (the 
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“guest memory” box in Figure 1). Note that kvm_create() does 
not allocate memory for the virtual machine. 

To create the first virtual CPU, use kvm_create_vcpu() with 
a value of O for the slot parameter—versions less than 65 
create the first virtual CPU during the call to kvm_create(). 

There are several methods to allocate memory for the 
virtual machine—for example, kvm_create_phys_mem(). The 
second argument of kvm_create_phys_mem() is the starting 
physical address of the requested region in the guest memory 
(in the pseudo-“ physical memory” of the virtual machine, not 
in the physical memory of the host). The third argument is the 
length, in bytes, of the region. The fourth indicates whether 
dirty page logging should be activated in the requested region, 
and the fifth argument indicates whether the pages may be 
written. On success, it returns the location of the allocated 
memory area as an address in the virtual address space of 
the calling process. 

Invoke the functions of Listing 1 within the same KVM 
context to create your first virtual machine, and execute it 
with kvm_run(). This function will return only if an I/O handler 
pointed in my_callbacks returns a nonzero value or an excep- 
tion occurs that neither the guest OS nor KVM can handle. 

Listing 3 contains the code for the launcher, including 
the load_file() function to copy the guest kernel image from 
a file to the virtual machine's memory space. Why is this 
image copied at offset OxfO000 of the guest's memory 
space? Because of the way real-mode works, as explained 
in the next section. 


16-Bit Real-Address Mode 

Processors compatible with the x86 architecture can support 
different operating modes. Two of them are 16-bit real-address 
mode. The most frequently used, these days at least, is 32-bit 
protected mode. The processor starts in real-address mode 
after a power-up or reset (so platform initialization code has 
to be written for this mode) and jumps to the instruction at 
address OxFFFFO. Usually, the BIOS’s initialization routine is 
located here. The first instruction of our simple kernel will 
be located there to take control of the platform as soon as 
it boots. Although with KVM it is possible to start a virtual 
machine directly in protected mode, our launcher won't 
do that in order to learn how to manipulate a PC just after 
power-up. 

The 16-bit real-address mode is a legacy mode inherited 
from the Intel 8086 processor, which is able to address up to 
1Mb of memory. 1Mb is 27° bytes, so addresses require 20 
bits. Given that the 8086's registers are only 16-bit wide, 
addresses are built by pairing two values. The first value is 
used as a selector (stored in a segment register), and the second 
value is used as an offset. With these, physical addresses are 
computed by the formula: 16 * selector + offset. 

For example, the selector:offset OxDEAD:OxBEEF represents 
the physical address OxEAQBF. To multiply the selector (OxDEAD) 
by 16, simply add a 0 to the right side of the number (OxDEADO). 
The addition then becomes the following: 


OxDEADO 
+ OXxOBEEF 


OxEAQBF 


Listing 3. Our First Virtual Machine Launcher (launcher.c) 


#include <stdio.h> #define MEMORY_SIZE (0x1000000) 16 Mb */ 
#include <stdlib.h> #define FIRST_VCPU (0) 
#include <unistd.h> 
#include <sys/types.h> int main(int argc, char *argv[]) 
#include <sys/stat.h> { 
#include <fcntl.h> kvm_context_t kvm; 
#include <libkvm.h> void *memory_area; 
/* callback definitions as shown in Listing 2 go here */ /* Second argument is an opaque, we don't use it yet */ 
kvm = kvm_init(&my_callbacks, NULL) ; 
void load_file(void *mem, const char *filename) if (!kvm) { 
{ fprintf(stderr, "KVM init failed"); 
int. ids exit(1); 
ill, nies } 
if (kvm_create(kvm, MEMORY_SIZE, &memory_area) != 0) { 
fd = open(filename, O_RDONLY) ; fprintf(stderr, "VM creation failed"); 
if (fd == -1) { exit(1); 
fprintf(stderr, "Cannot open %s", filename) ; } 
perror ("open") ; #ifndef KVM_VERSION_ LESS THAN_65 
exit(1); if (kvm_create_vcpu(kvm, FIRST_VCPU)) { 
} fprintf(stderr, "VCPU creation failed"); 
while ((nr = read(fd, mem, 4096)) != -1 && nr != 0) exit (1); 
mem += nr; } 
#endif 
if (nr == -1) { memory_area = kvm_create_phys_mem(kvm, ©, MEMORY_SIZE, 0, 1) 
perror("read") ; load_file(memory_area + Oxf0000, argv[1]); 
exit(1); 
} kvm_run(kvm, FIRST _VCPU) ; 
close(fd) ; 
} return 0; 
} 
Note that given a fixed value for the selector, it is possible 1Mb ; — 
to reference only 64Kb of memory (the offset’s allowed range). Oxtfffo 


Programs bigger than 64Kb must use multi-segment code. We 

will keep our kernel simple and make it fit into a single 64Kb 

segment. Our launcher will put the kernel image in the last 

segment (where the OxFFFFO entry point resides). The last segment 

starts at OxFOOOO as shown by the following calculation: Oxf0000 


kernel (64Kb) 


start() 


Start of the last segment 
= (Maximum 8086 Memory) - (Segment Size) 
= 1MB - 64KB 
= 0x100000 - 0x10000 = OxF0000 


A memory map of this is shown in Figure 2. 


Our 16-Bit Real-Address Mode Kernel 
We now can write a kernel in assembler with its first instruction 
at offset OxFFFFO. Note that unlike many processors, the 
x86 processor does not have a reset “vector”. It does not 
use the value at OxFFFFO as the location of the reset code; 
rather, it begins executing the code that is at OxFFFFO. 
Therefore, the “normal” code to place at OxFFFFO is a jump 
to the actual reset code. 
Our first kernel is shown in Listing 4. It merely sets the AX 0x00000 


Figure 2. Real-Address 
Mode Memory Map 


www.linuxjournal.com october 2009 | 63 


FEATURE Linux KVM as a Learning Tool 


Listing 4. kernel1.S 


.codel6 // Generate 16-bit code 
start: // Kernel's main routine 
xor %ax, %ax 
i; 
jmp 1b // Loop forever 
. = Oxfffo // Entry point 


1jmp $O0xf000, $start 


register to 0 and then loops forever. 
In the second to the last line, the dot (.) refers to the 
current location counter. Therefore, when we write: 


2 = Oxi hho 


we instruct the assembler to set the current location to 
address OxFFFO. In real-mode, address OxFFFO is relative to 
the current segment. Where does the segment offset get 
specified? It comes from the call to load_file() in Listing 3. 
It loads the kernel at offset OxFOOOO. This, combined with 
the assembler offset, will place the ljmp at address OxFFFFO, 
as required. 


How to Build It 
The kernel binary should be a raw 64Kb 16-bit real-address 
mode image, and not a normal ELF binary (the standard binary 
format used by Linux). To do this, we need a special linker 
script. We use GNU Id for this, of course, which accepts script 
files to provide explicit control over the linking process. 

A linker is a program that combines input binary files into a 
single output file. Each file is expected to have, among other 
things, a list of sections, sometimes with an associated block 


Although with KVM it is possible 
to start a virtual machine 
directly in protected mode, our 
launcher won't do that in order 
to learn how to manipulate a 
PC just after power-up. 


of data. The linker’s function is to map input sections into 
output sections. GNU Id uses, by default, a linker script 
specific for the host platform, which you can view by using 
the -verbose flag: 


$ gcc -W1,-verbose hello-world.c 
To build our kernel, we don’t use the default script but 
instead the simple script kernel16.lds, shown in Listing 5. 


The SECTIONS command controls how to make the 
mapping and how to place the output sections in memory. 
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Listing 5. Linker Script kernel16.lds 
QUTPUT_FORMAT (binary) 


SECTIONS { 
oles 
PUCXte on 4s (cnt et(ebext)l a) 
. = ALIGN(4K); 
plete) RB Mf eXCaGleheey) 3 
. = ALIGN(16); 
SS 2 Hf (lass) F 
. = ALIGN(4K); 
.edata = .; 


Listing 6. Building a 16-Bit Kernel Image 


$ gcc -nostdlib -Wl,-T,kernel16.lds kernel1.S -o kernell 
$ 1s -oh kernell 
-rwxr-xr-x 1 djprotti 64K 2008-10-17 19:09 kernell 


Directives follow the syntax: 
.output-section : [optional-args] 
{ input-section, input-section, ... } 


The kernel16.Ids script sets the current location at offset 
Ox0. Then, the output .text section will start there and will 
contain the contents of any .init and .text input sections. 

Next, we align the current location to a 4KB boundary and 
create the .data and .bss output sections. Use kernel16.lds to 
generate the kernel image as shown in Listing 6. 

The -nostdlib flag avoids linking the standard system 
startup files and libraries (these will not be available inside 
our virtual machines). After this, we have our 64Kb 16-bit 
real-address kernel image. 


How to Test It All 
The Makefile in Listing 7 contains the commands to build both 
the kernel and the launcher. 

Launch the virtual machine with kernel1 as guest with the 
following command: 


§ ./launcher kernell 


If everything goes well, you will see no output, and the 
guest kernel should be consuming all of its available CPU. If 
you run the top command in another console, and you see 
output similar to that of Listing 8 (100% CPU usage for the 
launcher process), you have your kernel running in your first 
KVM virtual machine! 


An Improved Kernel 
Now, let’s build a kernel that communicates with the world. First, 
choose one of the I/O ports and use it to implement a “serial 


port”. Name the chosen port as |O_PORT_PSEUDO_SERIAL 
(as shown in Listing 10), then modify the outb callback in 
the launcher to interpret bytes sent to this port as characters 


CHECK OUT OUR BUYER'S GUIDE ON-LINE. 
Go to www.linuxjournal.com/buyersguide where you can learn 
more about our advertisers or link directly to their Web sites. 


Thank you as always for supporting our advertisers by buying 


Listing 7. Makefile 

# If KVM was compiled from sources and you have errors about 
# missing asm/kvm*.h files, copy them from 

# kvm-XX/kernel/include/asm/* to {prefix}/include/asm/ 

CC=gcc 

KERNEL16_CFLAGS=-nostdlib -ffreestanding -W1,-T,kernel16.1ds 


alll: launcher kernell 


launcher: Launcher.o 
$(CC) launcher.o /usr/lib/libkvm.a -o launcher 


launcher .o: 


kernell: kernel1.S 
$(CC) $(KERNEL16 CFLAGS) kernel1.S -o kernell 


clean: 
rm *.o Launcher kernell 


Listing 8. Output of top While Our Launcher Is Running 


PID USER S %*CPU %MEM TIME+ COMMAND 
8002 djprotti R 100 06.8 1:53.19 launcher 
7428 Ciprotti 5 0 8.8 0:04.45 gnome-terminal 
8005 djprotti R 0 0.0 0:00.02 top 
1 root 5 0 0.0 Oses.92 iWimie 
2 root 5 0 @.0 0:00.00 kthreadd 
3 root s 0 0.0 0:00.12 migration/®@ 
4 root 5 0 0.0 0:02.76 ksoftirqd/®@ 
5 root 5 0 0.0 0:00.01 watchdog/0 


Listing 9. Pseudo-Serial Port Implementation in launcher.c 
#include "runtime.h" 


static int my_outb (void *opaque, uintl6 t addr, uint8 t data) 
{ 
if (addr == I0 PORT PSEUDO SERIAL) 
if (isprint(data) || data == '\n') 
putchar (data) ; 
else 
putchar('.'); 
else 
printf("outb: %x, %d\n", addr, data); 
fflush (NULL) ; 


return Q; 
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Listing 10. kernel2.S 


#include "runtime.h" 


.codel6 

Sivalnites 
mov $0x48 ,%al Hi Mn 
outb %al,$10_PORT_PSEUDO_ SERIAL 
mov $0x65 ,%al // e 
outb %al,$10_PORT_PSEUDO SERIAL 
mov $Ox6c ,%al Hit Ah 
outb %al,$10_PORT_PSEUDO_SERIAL 
mov $Ox6c ,%al Hi 1 
outb %al,$10_PORT_PSEUDO SERIAL 
mov $Ox6f ,%al Hi © 
outb %al,$10_PORT_PSEUDO SERIAL 
mov $0x0a,%al // new_line 
outb %al,$10_PORT_PSEUDO SERIAL 


hit // halt the processor 


. = Oxfffo 
1ljmp $0xf000, $start 


Listing 11. runtime.h 


#i fndef 


#define 


_RUNTIME_H__ 
_RUNTIME_H__ 


// port to use for general purpose output 
#define IO PORT _PSEUDO_SERIAL Oxfl 


#endif /* __RUNTIME_H_ */ 


printed to a serial console, and redirect them to launcher’s 
standard output as shown in Listing 9. 

Then, build a second kernel (kernel2) whose only task is 
to print “Hello\n” to its pseudo-serial port and then halt, as 
shown in Listing 10. 


Build both the launcher and kernel2, and run them as usual. 


The output should be similar to this: 


$ ./launcher kernel2 
Hello 


Now the top command should show 0% CPU usage for 
the launcher process, because its virtual CPU is halted. 

As a last example, an improved kernel is shown in 
Listing 12, using the OUTSB string output instruction and 
the REP prefix to repeat it the number of times specified 
by CX. Interestingly, this code generates only one I/O exit 
to output the entire string. Compare this against the 
previous kernel2, which generates one |/O exit for each 
outb execution, with the associated overhead due to 
context switches. You can use the kvm_stat Python script 
from the KVM sources to see this and other behaviours 
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Listing 12. kernel3.S (output using OUTSB) 


#include "runtime.h" 


.codel6 

start: 
mov $(10_PORT PSEUDO SERIAL), %dx 
cs lea greeting, %si 


mov $14, %cx 
cs rep/outsb // kvm_stat reports only 
// *one* jo_exit using this 


hit 


align 16 
greeting: 
see Nello, Morriell! win" 
. = Oxtffo 
1ljmp $O0xf000, $start 


of the virtual machines. 
The CS prefix before the LEA and OUTSB instructions are 
needed to fetch data (greeting string) from the code segment. 


What's Next? 

At this point, you have the basis to experiment with all 
kinds of real-mode code. You can extend the examples to 
set an IDT and handle interrupts or add more I/O devices. 
A good starting point is interrupts to learn the constraints 
of interrupt context, and another one is to investigate the 
rest of LiDKVM's methods. 

However, real mode is not enough to learn all the things 
that current kernels do on the x86 platform. For this reason, in 
a follow-up article, we will extend our launcher a little in order 
to handle kernels running in 32-bit protected mode. This 
change will give us the ability to write kernels in the C language, 
allowing for rapid development of bigger kernels. It also will 
open the door for experimenting with segmentation, paging, 
privilege levels (two or more rings) and more. 

Remember, low-level system programming is a challenging 
task, but with Linux KVM, it can be easy. So, go ahead and 
code, have fun and you will learn a lot about how computer 
systems work in the process! 


Duilio Javier Protti (duilio.j.protti@intel.com) is a software engineer with Intel Corp., in 
Cordoba, Argentina. He currently is working on a team specializing in virtualization technology. 
Before joining Intel, he wrote LibCMT (a library for composable memory transactions), was the 
maintainer of the Infinity XMMS plugin and contributed to various open-source projects, such 
as Nmap, Libvisual and others. 


Resources 


A Good Book on PC Assembly by Dr Paul Carter: 
drpaulcarter.com/pcasm 


KVM Sources: sourceforge.net/projects/kvm 
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Android 
Everywhere! 


Put Android where you'd least expect it—from 
Phones to virtual machines. 


BILL CHILDERS 


t seems you can’t hit a tech news site or read a magazine these days without encountering 
some mention of Android. If you've not been keeping up on the news, Android is a Linux- 
based OS, designed by Google that’s geared to run on lightweight devices like cellular phones 
and Webpads. One of Android's key features is that developers can write code for the OS in 
Java, making it a very easy platform for developers to work with. 

The first Android-powered product was the T-Mobile G1, made by HTC and known as the Dream. 
The Dream has a 528MHz ARM11 CPU, 192MB of RAM and 256MB of Flash, so it’s a capable smartphone, 
and it's part of an open standards effort from the Open Handset Alliance and Google. As a result of 
Google's involvement, it’s been touted as “The Google Phone” by the press. 

For this article, | set out to see how many devices | could put Android on and how difficult each one 
was to get running. Because we’re hearing buzz about Android-powered Webpads, phones and even 
Netbooks, | wanted to discover just what the hype was about. | elected to skip past the gloss and dive 
into the OS itself and see exactly what it takes to get it running on a device. 
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Exploring Android on the G1 
The HTC Dream/T-Mobile G1 phone 
(Figure 1) comes in a developer version 
that allows unsigned binaries to be run, 
and it does a few other things that the 
regular G1 doesn’t do. Because | had a 
regular G1, | figured a good place to 
start my Android exploration would be 
to see if | could get the developer OS 
running on a release device. Not surpris- 
ingly, T-Mobile frowns upon anyone 
doing this and puts roadblocks in the 
device to prevent it from happening. 
Also surprising, it turned out to be really 
easy, as there are holes in the firmware 
that allow you to gain root access on 
the phone. 

Once you get root, you pretty much 
can do what you want to the device, 
including flashing the developer 
version of the OS. The “Hacking Your 
G1/Dream” link in the Resources section 
of this article contains the details, but 
basically the steps are mostly standard 
Linux command-line fare, taking advan- 
tage of a bug in the firmware where 


Figure 1. The HTC Dream/T-Mobile G1 Smartphone 


everything you type at the keyboard is 
sent to the OS. (Try typing reboot on 
an older G1 at any time. It will reboot 
spontaneously!) 

You most likely will have to down- 
grade your firmware to a version that 
has the known exploit, and then take 
advantage of the exploit to gain root, 
but once that’s done, you can reflash 
the device with any firmware you 
choose, using the standard update 
method. If you choose to do this, 
standard disclaimers and waivers apply 
about breaking your hardware (see the 
Disclaimers and Waivers sidebar), as 
you're definitely doing something that 
has the potential to turn your several- 
hundred-dollar smartphone into an 
expensive brick. If you do decide to 
do this, however, | recommend JF's 
excellent 1.51 ADP build, as that retains 
root capabilities and allows you to run 
unsigned binaries (see the link to JF’s 
Blog in the Resources section). 

Once the latest build of the OS 
(code-named Cupcake) is on the now- 
rooted phone, you 
can build your own 
binaries for it, if 
you're a coder 
type, or grab 
things others have 
done from the 
Internet. Of course, 
if you do down- 
load someone 
else’s binaries, 
standard dis- 
claimers apply 
there too. Can you 
imagine the data 
charges that could 
be possible if you 
had a rootkit or 
trojan on your 
always-connected 


Disclaimers and Waivers 


| can’t stress enough: take extreme care when hacking mobile devices. Besides 
the obvious peril of bricking the device and making it unusable, there is the even 
greater danger of causing yourself an extremely large cellular bill. This is a really 


good way to test to see if your “unlimited’ 


" data plan truly is unlimited. You'll 


find out the hard way that most aren't. Take precautions, and try to find hacks 
that other people have done and reported success doing. Unless you have 
unlimited funds, blazing a trail in this area can become expensive quickly. 
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Figure 2. Android Booting on a WinMo Phone 


mobile device? This is exactly why 
T-Mobile doesn't want the devices 
hacked, as it could congest its network. 


Going Boldly Where No One 
Has Gone Before... 
As getting control of a G1 was relatively 
easy, | started wondering about 
installing Android on other devices. 
A quick scan of my desk revealed an 
unused AT&T Fuze cell phone, other- 
wise known as the HTC Touch Pro 
(code-named Raphael100). The HTC 
Touch runs Windows Mobile, not 
Android, but the units are both made 
by HTC and seemed to have similar 
hardware. | began to wonder if it would 
be possible to run Android on that 
phone, because they had the same 
manufacturer. 

| started researching the feasibility of 
running Android on the Touch Pro, and | 
discovered that a group of enterprising 
developers already had done this very 
thing. Luckily for me, they made their 
distribution available as well (see 
Resources), so getting Android running 


Figure 3. Kernel Messages on Windows Mobile? 


on the HTC Touch Pro was almost 
as easy as getting it going on the 
G1 (Figure 2). 

As it turns out, getting Android 
running on the Touch Pro was as easy as 
downloading a .zip file of the distribu- 
tion and unzipping the contents of that 
file to a MicroSD card. Once that was 
done, | put the card into the phone and 
used the Windows Mobile file manager 
to navigate to a directory on the card 
called tmp. Within that directory was a 
program called haret.exe. | ran that, and 
the screen on the phone went black, 
and then it showed me the familiar 
Linux kernel messages as it began to 
boot Android (Figure 3). 

Just like the loadlin days, when a 
DOS program could bootstrap the Linux 
kernel into booting, haret.exe boot- 
straps Android from the Windows 
Mobile environment. Before long, | 
was greeted with the Android desktop 
environment. However, all was not right 
with this port of Android. Although | 
could launch some of the applications, 
like the contact manager and browser, 


Where Are You Using Android? 


We want to know what devices you're running Android on. Share with our editors 
in the Android Everywhere forum at www.linuxjournal.com/forums/hot-topics 
where author and LJ Virtual Editor Bill Childers will be moderating. 
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the 3G modem inside the phone was 
not operational, nor were the micro- 
phone or speaker. About the only thing 
| could do was send and receive SMS 
messages, though it did do that exactly 
like the G1. Yes, just like the early days 
of Linux, it seems that device drivers for 
various pieces of hardware don’t exist or 
don’t work properly. However, this is a 
rapidly moving target, and the Android 
developers are working hard to make 
progress in this area. 


Android on a Netbook—Is It 
Usable? 

Because | had such relative success with 
Android on a Windows Mobile device, | 
proceeded on to see what it would take 
to get it running on my Netbook, an 
Acer Aspire One. | found that there is 

a project underway to port Android to 
x86 platforms, so | started to investi- 
gate. This seemed semi-straightforward, 
even though it required me to custom 
build the distribution myself. 

The x86 porting project allows you 
to build either a VirtualBox virtual 
machine or an installer for an ASUS Eee 
PC as the target. | found the main 
Android code repository, which had 
excellent instructions on how to set 
up your build environment and get 
all the various libraries installed (see 
Resources). If you're running Ubuntu, 
all that’s needed is a simple apt-get 
statement, and all the build dependencies 
and libraries are installed in a snap. 
However, once you've done that, you 
shouldn’t get the source from the main 
Android repository. 

Although you can get it from the 
main repository and patch it yourself, 
there is a prepatched source tree avail- 
able at the x86 porting site via SVN (see 
Resources). | spent a lot of time trying 
to patch the main source of Android for 
running on x86 only to have it fail near 
the end of the three-hour-long build 
cycle, or worse, fail to boot the OS 
image after it reported a clean build. 

The prepatched code built correctly 
on the first try, and | was able to get 
the VirtualBox virtual machine going 
with little hassle (Figure 4). One thing | 
did discover is that the VirtualBox virtual 
machine failed to boot unless the VM 
was configured to have a serial port. 

Once my Android VM was up and 
running, | started playing with it. Unlike 
the HTC Touch Pro port, this port had 
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Figure 4. Android Booting in a VM! Figure 5. Linux Journat—Android Style 

full networking. The first thing | did was see any performance differences, but enough from the virtual machine, 
fire up the browser (Figure 5) and was there were enough hardware incom- and going through another round of 
that ever surprising! It's possibly the patibilities that | reached the point of extensive troubleshooting just wasn’t 
fastest browser I’ve ever seen. Granted, diminishing returns. | figured I'd learned worth the effort. 


it’s optimized for a 5OOMHz ARM, and | 
was running it on a 2GHz dual-core 
CPU, but it was lightning fast. It’s still a 
mobile browser, however, so it's not full- 
featured by any stretch of the word. 
However, aside from surfing the 
Web, mobile style, and playing with the 
Terminal application, there wasn’t much 
of interest with Android on a VM. The 
applications aren't compelling enough 
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FEATURE Android Everywhere! 


Conclusion 

My exploration of Android was a fun and educational exercise, 
even if it was very frustrating at times. Android proves that 
Linux has come a long way from the days of the Sharp Zaurus 
on mobile devices, and on the T-Mobile G1, it's smooth, fun 
and easy to use. The G1 definitely is a contender for top 
smartphone, and when you're competing with the likes of the 
iPhone and BlackBerry, that’s saying a lot. 

Android as an alternative to Windows Mobile holds a 
lot of promise. Its open architecture means it can take 
advantage of the Open Source movement to roll in new 
features, and it can give many current Windows users their 
first taste of how sweet Linux and open source can be. 
Unfortunately, it’s not ready for prime time now, but that'll 
change as developers figure out the hardware and get 
driver support for the various handsets—much like Linux's 
desktop support has grown during the past few years. This 
is a moving target, and a lot can change quickly. | plan to 
keep an eye on this space for more news on Android on 
formerly Windows Mobile handsets. 

Android on a Netbook, however, gives me pause. After 
spending some time with the VirtualBox virtual machine, | 
can't really see how this is better than, say, Ubuntu Netbook 
Remix or even regular Ubuntu. Android is optimized for small 
touchscreens and tiny CPUs. Most Netbooks have a lot more 
CPU than is required to run Android, yet they don’t have the 
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Breaking News 


Just before press time, a team of Android developers 
released a live CD for Android. If you want to test drive 
Android on your Netbook, laptop or even a virtual 
machine, it’s extremely easy now. Simply download the 
image, burn it to CD, and boot from it—no compiling 
needed. Check it out—you may get hooked on it enough 
to buy a G1 phone! 


touchscreen. Android is designed for use with a minimal or 
onscreen keyboard, and Netbooks have full, functioning 
keyboards. After fiddling with it, the use case for Android on 
a Netbook just doesn’t seem very compelling. | think there’s 
more value in using a Netbook as a tiny, full-featured laptop 
than using it as a large mobile Internet tablet. However, don’t 
let my opinion sway you. Go test it for yourself! You'll learn a 
lot, and if you have the skills to improve the port, contribute 
some fixes. The developers probably would appreciate the 
help, and you'd be contributing to something that could touch 
a lot of people.m 


Bill Childers is an IT Manager in Silicon Valley, where he lives with his wife and two children. He 
enjoys Linux far too much, and probably should get more sun from time to time. In his spare time, 
he does work with the Gilroy Garlic Festival, but he does not smell like garlic. 


Resources 


The Open Handset Alliance: 
www.openhandsetalliance.com/android_overview.html 


JF’s Blog (Android Developer Extraordinaire): jf.andblogs.net 


Hacking Your G1/Dream: forum.xda-developers.com/ 
showthread.php?t=442480 


Android for the HTC Touch Pro: connect-utb.com/ 
index.php?option=com_rokdownloads&view= 
folder&ltemid=68&id=3:htc-raphael 


Porting Android to x86: code.google.com/p/ 
patch-hosting-for-android-x86-support 


Setting Up Your Machine to Build Android: 
source.android.com/download 


Getting the Patched x86 Version of Android: svn checkout 
http: //patch-hosting-for-android-x86-sup- 
port.googlecode.com/svn/trunk/ patch-hosting-for- 
android-x86-support-read-only 


Brock Tice’s Eee PC Build of Android: virtuallyshocking.com/ 
2008/12/20/building-android-for-the-asus-eeepc-701 
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Clutter 


Use Clutter to develop OpenGL applications with rich 2-D and 3-D interfaces that 
include object rotation, scaling, texturing and more. ALEX CRITS-CHRISTOPH 


Meet one of the most revolutionary 
toolkits available for Linux: Clutter. Clutter 
is an OpenGL-based toolkit, described as 
an “open-source software library for 
creating fast, visually rich and animated 
graphical user interfaces”. Clutter provides 
a simple API for powerful three-dimensional 
and two-dimensional manipulation. 
Creating interactive games, 3-D media and 
animated applications for Linux systems 
with Clutter is cleaner, easier and quicker 
than coding an OpenGL application with 
more conventional methods. 

Clutter comes with many built-in tools 
and effects. Rendering object rotation, 
scale, texture and opacity are built right in 
and can be accomplished with a few lines 
of code. Rendering and controlling the 
GStreamer multimedia API also is easy 
with an additional library. There even are 
Webkit bindings, so manipulating Web 
pages in a Clutter program is simple. 

Clutter has been used in many 
successful applications and open-source 
projects. Take, for example, the open- 
source Elisa media center. Developed by 
Fluendo, Elisa is a 3-D media center— 
one of the most sophisticated alterna- 
tives to software such as Windows 
Media Center available for Linux. Elisa 
makes use of Clutter’s animation and 
3-D API in its elegant interface. 

The Ubuntu Mobile Internet Device 
Edition, developed by the Ubuntu Mobile 
community, also uses Clutter for its main 
user interface. Additionally, the Moblin 
Project plans to use Clutter in its software 
platform. Clutter’s use is widespread 
across Linux systems and is becoming 
more and more popular every day. 

Installing Clutter on Linux systems is 
extremely easy with the use of binary 
package managers. Install Clutter, the 
Cairo add-on, the GStreamer add-on and 
the Python bindings. Using your distribution’s 
package manager, install the following 


packages: libClutter, libClutter-cairo, 
libClutter-gst and python-Clutter. 

Different distributions will have 
different versions available, and it is 
recommended that you install the latest 
possible version. However, you need 0.8.0 
or 0.8.2 of the libClutter packages to follow 
the examples and run the code given in 
this article. If the version number in your 
package manager is different from either 
0.8.0 or 0.8.2, you should install Clutter 
from source. See Resources for the URL 
to Clutter’s source files. 

In this article, I’m using Clutter’s 
Python bindings to work with Clutter. 
More can be done with Python in just a 
few lines of code, so using that language 
makes it easier to explore and understand 
Clutter. To test your install of Clutter, 
simply run Python and do the following: 


import clutter 


If you get a blank prompt back 
with no errors, the Clutter module 
was imported successfully, and you've 
installed python-Clutter correctly. 

Now, let's start with a simple “Hello 
World!” Clutter application with Python. 
You probably should turn off any desktop 
effects or compositing window managers, 
such as Compiz Fusion. Most Linux video 
drivers will not allow multiple OpenGL 
or 3-D processes to run simultaneously 
with a compositing window manager, 
which includes Clutter, because of its 
3-D capabilities. 

To start the program, you need to 
import the Clutter module and define your 
main Class and an initialization function. 
Create a stage in the initialization func- 
tion. The stage is the base of any Clutter 
interface. On the stage, objects called 
actors can be seen and manipulated. 
Clutter uses the term actors to describe 
any objects that exist on the stage. 
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opensuse.png 


Figure 1. Elisa, Fluendo’s media center appli- 
cation, utilizes Clutter’s advanced library to 
add 3-D animation to its GUI. 


The sample program is shown in 
Listing 1, and the output window is 
shown in Figure 2. 

After creating the stage, set the stage’s 
properties and some properties of the 
window containing it. 

Set color of the stage to black by 
accessing Clutter’s predefined colors 
using Clutter.color_parse(). 

Next, set the size of the stage, which 
will set the size of the window. Also set 
the title of the window. 

To show our “Hello world” message 
on the stage, you need to create an 
actor—in this case, a label. Set the font 
type of the label, the text to display, and 
the color of the label. Here, let’s set the 
color manually rather than using a 
predefined color. Once the label is set, 
add the actor (the label) to the stage. 

Labels work similarly to GTK+ widgets, 
but Clutter is not widget-based in the 
same way GTK+ is. Although both have 
similar functions and parts, Clutter con- 
tains only a handful of built-in “widgets”, 
which are called actors. Clutter’s actors are 
limited to rectangles, labels, images, video 
textures and a few other items. 

To finish the example, tell the stage to 
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What’s Wrong with Compiz Fusion? 


Compiz Fusion is an OpenGL compositing 
window manager, capable of delivering 
3-D and smooth animation to the desktop. 
It is widespread on Linux desktops and 
is available on almost all modern Linux 
distributions. Often, Compiz Fusion is 
enabled by default. 


However, Compiz Fusion does not play nice 
with Clutter. In fact, on almost all video 
cards, most OpenGL- or SDL-based applica- 


show all of its contents and call the main 
Clutter loop, which will display the interface. 
The last step is to tell Python to create an 
instance of your class. 

Now, let's take a look at a more useful 
Clutter program. Our program will use the 
Clutter GStreamer library to display and 
control a video file on the stage. Connect 


Listing 1. “Hello World” Using Clutter 
import clutter 


class HelloWorld: 
Cer ime (Seine 


# Create stage and set its properties. 


self.stage = clutter.Stage() 
self 
self.stage.set_size(500, 400) 
self 


# Create Label and set its properties. 
color = clutter.Color(@xff, Oxcc, Oxcc, @xdd) 


he 
he set_font_name('Mono 32') 


o = clutter.Label() 
OF 

hello.set_text("Hello There! ") 
OF 
Oo. 


he 
he 


set_color(color) 
set_position(100, 200) 


# Add label to stage. 
self.stage.add(hello) 


# Start main clutter loop. 


self.stage.show_all() 


clutter.main() 


# Run program. 
main = HelloWorld() 


.Stage.set_color(clutter.color_parse('Black')) 


.Stage.set_title('Clutter Hello World') 


tions are slow and prone to flickering if 
Compiz is running. Some of the programs 
affected include Google Earth, Blender and 
most 3-D games. 


This is due to the X Window System's inability 
to render OpenGL applications along with 

a compositing window manager, such as 
Compiz Fusion, simultaneously on most 
video cards. However, DRI2 aims to fix this 
problem. DRI2 (Direct Rendering Infrastructure 


GStreamer’s video output to a video texture, 
which is then displayed on the stage and 
can be manipulated as an actor. 

There are three main actors on the 
stage: the play button, the pause button 
and the GStreamer video. When the pause 
button is pressed, the video pauses, 
and it will continue playing when the play 
button is pressed. 
The program is 
shown in Listing 2, 
and the window is 
shown in Figure 3. 

After the initial 
setup, define a sig- 
nal for responding 
to mouse clicks. 
Clutter uses signals 
to respond to events 
in the same way 
GTK+ does. The 
signal dictates that 
when any button 
on the mouse is 
clicked, the specified 
function is called. 

Next, create the 
buttons by creating 
rectangle actors for 
the button shape 
and text actors 
for the button text. 
Remember that 
Clutter is not 
widget-based, and 
there are no default 
button widgets as 
part of the API. 

To create the 
video, you need a 
video texture. A 
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2) will allow several OpenGL applications to 
run at once by directly rendering redirected 
windows. In time, DRI2 will ship along with 
most X.Org video drivers. 


With the implementation of DRI2 in most 
video drivers, the X Window System finally 
will be able to handle OpenGL with a 
compositing manager. However, currently 
on most video cards and using most drivers, 
Clutter conflicts with Compiz Fusion. 


video texture is a physical plane on which 
GStreamer can display the video. The 
video texture can be manipulated on 
the stage like an ordinary actor. 

You also need two other GStreamer 
elements to play the video: a playbin and 
a pipeline. Clutter-Gstreamer will play any 
file type that GStreamer can play. After 
creating the playbin, set the location of 
the video to play and add the playbin to 
the pipeline. 

Then, set the position of the video 
texture, add it to the stage and tell 
GStreamer to start playing the video. 

Finally, create the mouseClick function. 
This function is called when the mouse is 
clicked anywhere on the stage. Check to 
see if the left mouse button was clicked 
inside one of the buttons and if it was, 
change the size and color of the buttons 
to give visual feedback of the click. Tell 
GStreamer to start or stop the video 
depending on which button was pressed. 

An important feature of Clutter is 
its animation API. Animating any actors 


CHPPP ele wana: 


Hello There! 


Figure 2. The simple “Hello World” Clutter 
Program Running on Ubuntu 


step function causes the animation to 
speed up, then slow down, come to a 
halt, and then start up again. Clutter has 
several functions built in that you can use 
with the alpha, including sine, exponential 
and ramp functions. 

Next, define the Rotation behavior 
the animation uses. Clutter uses behavior 
effects to describe animations. The 
Opacity behavior, for example, can 
change the visual alpha of an actor, 
making it transparent or opaque. Other 

; behaviors include Scale, Path, Depth, 
Figure 3. Clutter Video Player Playing the Figure 4. The 3-D Clutter Video Player, B-Spline and Ellipse. 
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Monty Python Skit “How Not to Be Seen” Partway through Rotating the Video In this example, we tell our Rotation 
behavior to rotate over the x-axis, 
on the stage is easy with Clutter. Using Timelines are used in Clutter to control rotate clockwise, start rotating at an 
the animation API, you can add smooth animation and time events. The example angle of zero, end at 360, and finally, 
animations and effects to your Clutter timeline lasts for 100 frames at ten tell it to use the alpha created earlier, 
application. frames per second, and the timeline is respectively. After that, the rotational 
In the next example, let's take the set to loop forever. center, or the point the actor rotates 
GStreamer video texture and manipulate it Next create an alpha (see The Alpha around, is set to the approximate center 
in three dimensions. The GStreamer texture Functions sidebar) for the animation, of the GStreamer texture and the 
is rotating on the y-axis constantly. assign it to your timeline and give it a Rotation behaviour is applied to the 
The program is shown in Listing 3, and smooth step decreasing function. In sim- video texture. 
the window is shown in Figure 4. plest terms, the alpha is used to control In addition to the normal startup steps, 
After the initial setup, create a timeline. the speed of the animation. The smooth the timeline must be started. 
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Hopefully, you've learned a good deal start developing and programming using you've seen here, you'll be able to create 
about how Clutter works, and you can the Clutter API. Using just the features any interface that uses text, buttons, 


Listing 2. Clutter-Based Video Player 
# Specify video file to play. 
import clutter movfile = "file:///home/user/Videos/Video.mov" 
import gst playbin.set_property('uri', movfile) 
from clutter import cluttergst 
# Add to playbin to the pipeline. 


class HelloWorld: self.pipeline.add(playbin) 
Get 2 iit sel ty: 
# Create stage and set its properties. # Set position and start playing the video. 
self.stage = clutter.Stage() video_tex.set_position(90, 100) 
self.stage.set_color(clutter.color_parse('Black') ) self.stage.add(video_tex) 
self.stage.set_size(500, 400) self.pipeline.set_state(gst.STATE_PLAYING) 


self.stage.set_title('Clutter Basic Video Player') 

self.stage.show_al1() 

# Create signal for handling mouse clicks. clutter.main() 
self.stage.connect('button-press-event', self.mouseClick) 

def mouseClick (self, stage, event): 


# Create play button shape. # Mouse click function, called when the moused 
self.playBtn = clutter.Rectangle() # is clicked *anywhere* on the stage, we check 
self.playBtn.set_color(clutter.Color(66, 99, 150, @x99)) # the mouse coordinates manually to see if the 
self.playBtn.set_size(50, 30) # click occurred inside a button. 


self.playBtn.set_position(118, 34) 
self .stage.add(self.playBtn) # Check for left mouse button. 


if event.button == 
# Create play button text 


# and overlay the rectangle. # Check to see if stop button was pressed. 
playTxt = clutter.Label() if event.x > 218 and event.x < 268 and \ 
playTxt.set_text ("Play") event.y > 34 and event.y < 64: 


playTxt.set_color(clutter.color_parse('Black')) 
playTxt.set_position(130, 40) 


self .stage.add(playTxt) self.stopBtn.set_color (clutter .Color (33,50, 150, 0x89) ) 
self.playBtn.set_color (clutter .Color (66,99, 150, 0x99) ) 

# Same for stop button. self.stopBtn.set_size(49, 29) 
self.stopBtn = clutter.Rectangle() self.playBtn.set_size(50, 30) 
self.stopBtn.set_color(clutter.Color(66, 99, 150, 0x99)) self.pipeline.set_state(gst.STATE_PAUSED) 
self.stopBtn.set_size(50, 30) 
self.stopBtn.set_position(218, 34) # Check to see if the play button was pressed. 
self .stage.add(self.stopBtn) if event.x > 118 and event.x < 168 and \ 


event.y > 34 and event.y < 64: 
StopTxt = clutter.Label() 
StopTxt.set_text ("Pause") 


StopTxt.set_color(clutter.color_parse('Black')) self.playBtn.set_color (clutter .Color (33,50, 150, 0x89) ) 

StopTxt.set_position(225, 40) self.stopBtn.set_color (clutter .Color (66,99, 150, 0x99) ) 

self.stage.add(StopTxt) self.playBtn.set_size(49, 29) 
self.stopBtn.set_size(50, 30) 

# Create video texture. self.pipeline.set_state(gst.STATE_PLAYING) 

video_tex = cluttergst.VideoTexture() 

self.pipeline = gst.Pipeline("mypipe") # Run program. 

playbin = video_tex.get_playbin() main = HelloWorld() 
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images and video with Clutter. Of course, after learning the 
basics, the more advanced U! elements will become easier to 
understand and work with. 

In the future, the Clutter developers will continue to improve 
and update the API, and many new improvements are expected in 
the Clutter 1.0 release. You can learn more about the Clutter devel- 
opment process from the Web site (see Resources). Clutter is going 
to power many innovative open-source applications in the future.= 


The Alpha Functions 


At first, especially to those who've forgotten their Calculus 
and Algebra, the alpha functions may seem unpredictable or 
confusing. There is a large list of the number of available 
functions: exp_dec_func, exp_inc_func, ramp_dec_func, 
ramp_func, ramp_inc_func, sine_dec_func, sine_func, 
sine_half_func, sine_inc_func, smoothstep_dec_func, 
smoothstep_inc_func and square_func. Here's a brief 
explanation of each type: 


®@ Exponential functions: depending on whether you're 
using a decaying function or an increasing function, 
exponential functions make the animation speed up or 
slow down at an exponential rate. 


®@ Ramp functions: ramp functions animate at a constant 
speed. However, the full ramp function animates at 
both a negative and a positive constant speed by 
switching directions. 


® Sine functions: sine functions make the animation 
reverse. Like the graph of a sine function, the animation 
would speed up, slow down, change directions, speed 
up in the reverse direction, and then slow down again. 


® Smooth step functions: the smooth step function works 
logistically. It starts slowly, then quickly increases and 
finally slows down toward the end of the animation. 


™@ Square functions: square functions follow a step pattern, 
which results in quick changes between two constant 
animation speeds. 


Resources 


Clutter Home Page: clutter-project.org 
Elisa Project Page: elisa.fluendo.com 


Clutter Source Files: www.clutter-project.org/sources 


Listing 3. Clutter Rotating a Video 


import clutter 
import gst 
from clutter import cluttergst 


class HelloWorld: 
olay _— ulmiie_ _ (Geils 
self.stage = clutter.Stage() 
self.stage.set_color(clutter.color_parse('Black')) 
self.stage.set_size(500, 400) 
self.stage.set_title('Clutter 3-D Video Player') 


# Setup video. 

video_tex = cluttergst.VideoTexture() 
self.pipeline = gst.Pipeline("mypipe") 
playbin = video_tex.get_playbin() 

movfile = "file:///home/user/Videos/Video.mov" 
playbin.set_property('uri', movfile) 
self.pipeline.add(playbin) 
video_tex.set_position(90,80) 
self.stage.add(video_tex) 

self .pipeline.set_state(gst.STATE_PLAYING) 


# Create timeline that lasts for 100 frames 
# at ten frames per second. 
timeline = clutter.Timeline(100, 10) 


# Set timeline to loop forever. 
timeline.set_Loop(True) 


# Create an alpha. 
alpha = clutter.Alpha(timeline, clutter.smoothstep_dec_func) 


# Set up rotation. 

Rotation = clutter.BehaviourRotate( 
axis=clutter.Y_AXIS, 
direction=clutter .ROTATE_CW, 
angle_start=0, 
angle_end=360, 
alpha=alpha) 

Rotation.set_center(160, 160, 0) 

Rotation. apply (video_tex) 


# Start it all up. 
timeline.start() 
self.stage.show_all() 
clutter .main() 


# Run program. 
main = HelloWorld() 


Alex Crits-Christoph has been working with Linux for some time now. He enjoys developing and 
designing open-source graphical user interfaces. 
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EOF 


Turning the Internet 


Outside In 


Let’s hack an open Internet, starting at home. boc SEARLS 


You can only hack what's hackable. We owe 
Linux to the fact that operating systems are 
hackable, and that they can run on common 
hardware, much of which is also hackable. 
We also owe Linux to the Internet, which is a 
hack on wiring and data trafficking. 

For PCs and mobile devices, Linux is a 
defaulted choice. It's at GandhiCon 4. That 
and the first three GandhiCons are implicit in 
the Mohandas Gandhi quote, “First they 
ignore you, then they laugh at you, then 
they fight you, then you win.” 

The Internet, however, is another matter. 
The Internet Protocol (IP) arrived at 
GandhiCon 4 by 1981 (with IPv4). That's 
because it was created as what we might call 
a public protocol, connecting devices using 
just about any kind of network wiring, hard- 
ware and data link protocols (Ethernet, Token 
Ring, FDDI and so on), without prejudice. This 
made it easy and cheap for anybody to use. 

By design, the Internet Protocol was 
decentralized. It reduced network complexity 
inside the network as far as possible, while 
relying on intelligence at its end nodes. It was 
even agnostic toward addressing schemes, 
leaving choices up to implementations at 
higher levels in the stack and resolution up 
to the Address Resolution Protocol (ARP). 
Alas, what most people know best about 
the Internet is not its decentralized, depoliti- 
cized and free (as in both freedom and beer) 
public nature, but rather its centralized, 
politicized and costly (as in both freedom 
and beer) private one. This is the Internet of 
domain names that are privately owned 
(actually, rented), controlled by a central 
naming authority (the Internet Corporation 
for Assigned Names and Numbers, or 
ICANN) and filled with “pipes” mostly 
owned by private interests and highly 
cartelized. This is not an Internet to which 
we can simply connect. Instead, it’s one we 
can “access” only through Internet Service 
Providers—a class of businesses that was 
born when small independent companies 
found ways to make the Internet available 
to anybody with a land line and has since 
become the tertiary service of phone and 


cable companies selling “broadband” or 
“high-speed Internet” as the third act 
ina “triple play”. 

Although the former Internet is 
hackable, the latter one is not. 

As it happens, I’ve been living in the 
hackproof hell of the private, centralized 
Internet for the last two weeks, during which 
ime my home connection here in Santa 
Barbara has been intermittently plagued by 
high latencies and packet losses. My ping and 
raceroute tests clearly isolate the problem 
somewhere between my cable modem and 
he first IP address my packets encounter: a 
gateway downtown that’s also owned by the 
cable company. Cable company technicians 
hat have come to my house (four so far) 
have excused from blame my cable modem 
and all wiring between it and the service 
pole. They know the problem is somewhere 
in their system. They still have not solved it, 
and neither can |, even with help from many 
friends far geekier than myself. 

So here is a radical proposition. Let’s 
build the Internet we want—a free, open 
and hackable Internet—from the outside in. 

This is something Bob Frankston has been 
advocating for many years. What Bob wants 
is simple connectivity between any points 
floating on the vast resource he calls our “sea 
of bits”. His latest label for this is “ambient 
connectivity”. In his essay “Opportunity for 
Innovation”, Bob writes, “Once we can 
assume connectivity we can start taking 
advantage of the opportunities. It’s not just 
about high-value applications like education, 
commerce and entertainment. It's about 
basic infrastructure. We won't discover the 
real value until we've had a chance to experi- 
ence ambient connectivity.” In a follow-up 
essay titled “Zero Marginal Cost”, he adds: 


The idea that we can create 
our own solutions using raw, 
unreliable bits is at the heart of 
the Internet’s generativity.... 


We've already seen the power 
of zero marginal cost. It was the 
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availability of unmeasured local 
phone service that gave the 
United States the lead in adopting 
the Internet in the 1990s. We 
rejected digital phone service 
because the phone companies 
chose to charge a premium for 
that service. We just worked 
around it using modems because 
there was zero marginal cost for 
using the existing infrastructure. 


Bob's model of the Internet is home 
networking, expanded outward through 
converging communities. In my interview 
with Bob for the March 2008 issue of Linux 
Journal (www.linuxjournal.com/article/ 
10033), he said, “The networks in our 
homes are a good example. You ‘just’ print 
without worry about negotiating for the 
printing provider.” 

As it happens, I’m also shopping for 
home networking gear—in particular, for 
a router/switch to connect the 16 Ethernet 
jacks scattered about the house. Cat-6 
wiring runs from each of those jacks to a 
patch panel in a wiring closet. The cable 
company’s modem is in there too. 

Lemme tell ya, if there’s a category 
ripe for disruption, it’s home networking. 
I've been looking at Belkin, Cisco/Linksys, 
D-Link, Netgear and others—none of 
which are especially helpful. The 8-port 
device I'm replacing is a Netgear 
router/switch that was billed as a “VPN 
Firewall” but failed at the essentials: its 
gears were stripped by the cable compa- 
ny’s new 20Mb downstream data speeds. 

So let’s look at making the Net 
hackable from the outside in. VCs always 
are asking about market size and “pain 
points” in need of relief. | can’t think of 
a bigger, or more ideally hackable, pain 
than the one we find right at home.m™ 


Doc Searls is Senior Editor of Linux Journal. He is also a 
fellow with the Berkman Center for Internet and Society at 
Harvard University and the Center for Information Technology 
and Society at UC Santa Barbara. 
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